A monthly thread for posting rationality-related quotes you've seen recently (or had stored in your quotesfile for ages).

  • Please post all quotes separately, so that they can be voted up/down separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)
  • Do not quote yourself.
  • Do not quote comments/posts on LW/OB.
  • No more than 5 quotes per person per monthly thread, please.
Rationality Quotes: November 2010
New Comment
367 comments, sorted by Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings
[-]anonym620

If you are in a shipwreck and all the boats are gone, a piano top … that comes along makes a fortuitous life preserver. But this is not to say that the best way to design a life preserver is in the form of a piano top. I think that we are clinging to a great many piano tops in accepting yesterday’s fortuitous contrivings.

Buckminster Fuller

Kołakowski's Law, or The Law of the Infinite Cornucopia:

For any given doctrine that one wants to believe, there is never a shortage of arguments by which to support it.

Leszek Kołakowski

3Drawbacks
If you set out to beat a dog, you're sure to find a stick. -- Old Yiddish Proverb
0wedrifid
I like it.
-2stochastic
+1
[-]Nisan290

Know the hair you have to get the hair you want.

-Pantene Pro-V hair care bottle

[-]anonym290

Go down deep enough into anything and you will find mathematics.

Dean Schlicter

[-]PeterS280

Rule I

We are to admit no more causes of natural things than such as are both true and sufficient to explain their appearances.

To this purpose the philosophers say that Nature does nothing in vain, and more is in vain when less will serve; for Nature is pleased with simplicity, and affects not the pomp of superfluous causes.

Rule II

Therefore to the same natural effects we must, as far as possible, assign the same causes.

As to respiration in a man and in a beast; the descent of stones in Europe and in America; the light of our culinary fire and of the sun; the reflection of light in the earth, and in the planets.

Rule III

The qualities of bodies, which admit neither intensification nor remission of degrees, and which are found to belong to all bodies within the reach of our experiments, are to be esteemed the universal qualities of all bodies whatsoever.

For since the qualities of bodies are only known to us by experiments, we are to hold for universal all such as universally agree with experiments; and such as are not liable to diminution can never be quite taken away. We are certainly not to relinquish the evidence for the sake of dreams and vain fictions of our own dev

... (read more)
[-]DSimon250

Man, I'm amazing! I'm a machine that turns FOOD into IDEAS!

-- T-Rex, Dinosaur Comics #539

I know this is well known, but to supplement the T-Rex:

A mathematician is a device for turning coffee into theorems.

-Alfréd Rényi/Paul Erdős

Yes, and don't forget the dual result that a comathematician is a device for turning cotheorems into ffee.

1SilasBarta
And a cat is a device for turning kibble into cuddle.

The fact that I have no remedy for all the sorrows of the world is no reason for my accepting yours. It simply supports the strong probability that yours is a fake.

H.L. Mencken, Minority Report.

6[anonymous]
I wrote about this. The idea is: I can criticize a plan that claims wonderful successes, even if I have no corresponding plan of my own. Maybe we don't know how to get wonderful successes at all. Maybe they're impossible. Maybe your reasoning is suspect.
2wedrifid
I am not sure I get it.

A more direct paraphrasing would be, Just because I don't have all the answers doesn't mean that your answers are correct.

A concrete example: just because scientists don't currently know everything about how evolution happened, that doesn't mean that Young Earth Creationists are right. Typical YEC debating strategy is to look for gaps (real or imagined) in our current theories, and act as if that proves that God created the world in six days, and from the dust created every creeping thing that creepeth upon the earth, etc.

-2Paul Crowley
No, it speaks of remedy. It's not about beliefs about the world, but about courses of action, and there he's dead wrong - a course of action can only be bad by comparison to a better alternative.

"We must do something. This is something. Therefore, we must do this." is a fallacy. (The Politician's Syllogism.) Mencken's statement pretty clearly includes the course of action of not taking action; he's stating that any action is not necessarily better than no action, and that taking on any belief is not necessarily better than holding no belief.

I don't think either of you are getting it right. I'm not familiar with the context of this particular quote, but knowing it's from Mencken, he's clearly referring to various idealistic busybodies and their grand (and typically disastrously unsound) plans to solve the world's problems. The quote is directed against idealists who assume moral high ground and scoff at those who question their designs.

2Paul Crowley
Ah, so it's about whether a plan meets some absolute standard, rather than which plan is best, and the moral is that just because I don't know of a plan that meets standard X is no reason to think your plan will - in fact the reverse.
6AlanCrowe
I think the absolute standard in question is the status quo. Will the proposed remedy make things worse? Mencken has no remedy of his own. In the first sentence he denies that this lack is evidence in favour of the proposition that somebody else's remedy will be an improvement on leaving things alone.
5Vladimir_M
Basically, yes. For instance, the alcohol prohibitionists of Mencken's day were a prime example of the sort of people he targeted with this quote.
5PhilGoetz
We can call a course of action bad if doing nothing is a better alternative.
1BillyOblivion
But sometimes that better alternative is "let's wait and see". And that's what many people aren't willing to do.
[-]MBlume240

When I was halfway through my Ph.D. I formulated a hypothesis: The proximate challenge that keeps you from graduating is that you have to write a thesis. But the ultimate challenge to getting your Ph.D. is this: You somehow have to learn to understand, deep down, that all your romantic notions about the Ph.D. are bunk, that you will be exactly the same person on the day after you get it that you were the day before, and that you need to stop waiting for the day when you feel like a god and just write something down and get on with life.

It may take you years to accept this, and it may drive you to drink, but after you get to that point you can graduate.

Only then will you be able to live with the fact that your thesis looks like crap to you. Your thesis will always look like crap to you. Either you will have figured out absolutely everything and your thesis will look incredibly boring to you, because you've moved on, or -- vastly more likely -- your thesis will look woefully incomplete because, geez, there is so much that you couldn't figure out, and you're just so stupid!

Or, most likely of all, you will think both of these things at the same time.

Similarly: Being the world's foremo

... (read more)
3Document
-- attr. Albert Einstein

From desert cliff and mountaintop we trace the wide design,
Strike-slip fault and overthrust and syn and anticline...
We gaze upon creation where erosion makes it known,
And count the countless aeons in the banding of the stone.
Odd, long-vanished creatures and their tracks & shells are found;
Where truth has left its sketches on the slate below the ground.
The patient stone can speak, if we but listen when it talks.
Humans wrote the Bible; God wrote the rocks.

There are those who name the stars, who watch the sky by night,
Seeking out the darkest place, to better see the light.
Long ago, when torture broke the remnant of his will,
Galileo recanted, but the Earth is moving still.
High above the mountaintops, where only distance bars,
The truth has left its footprints in the dust between the stars.
We may watch and study or may shudder and deny,
Humans wrote the Bible; God wrote the sky.

By stem and root and branch we trace, by feather, fang and fur,
How the living things that are descend from things that were.
The moss, the kelp, the zebrafish, the very mice and flies,
These tiny, humble, wordless things--how shall they tell us lies?
We are kin to beasts; no other answer can we bring.
The truth... (read more)

Long ago, when torture broke the remnant of his will, Galileo recanted, but the Earth is moving still.

What evidence is there that Galileo was tortured?

So far as I know, he wasn't, just placed under house arrest. It jumped out at me too; you really have to get these poems exactly right on a factual level or it takes a lot away.

The modern conception of Galileo as someone harshly prosecuted for his beliefs seems rather exaggarated: in reality, he was even explicitly encouraged to write a book on the subject by the church. It was only when he offended the Pope in his book that he got sent to house arrest.

In the end, Cardinal Bellarmine, acting on directives from the Inquisition, delivered him an order not to "hold or defend" the idea that the Earth moves and the Sun stands still at the centre. The decree did not prevent Galileo from discussing heliocentrism hypothesis (thus maintaining a facade of separation between science and that church). For the next several years Galileo stayed well away from the controversy. He revived his project of writing a book on the subject, encouraged by the election of Cardinal Maffeo Barberini as Pope Urban VIII in 1623. Barberini was a friend and admirer of Galileo, and had opposed the condemnation of Galileo in 1616. The book, Dialogue Concerning the Two Chief World Systems, was published in 1632, with formal authorization from the Inquisition and papal permission. [...]

Earlier, Pope Urban VIII had personally asked Galileo to give arguments for and against helio

... (read more)

So far as I know, he wasn't, just placed under house arrest.

According to Owen Gingerich's The Great Copernicus Chase, the 1633 decree calling Galileo to be interrogated* read, in part, as follows:

Galileo Galilei ... is to be interrogated concerning the accusation, even threatened with torture, and if he sustains it, proceeding to an abjuration of the vehement [suspicion of heresy] before the full Congregation of the Holy Office, sentenced to imprisonment....

(Emphasis added.) Gingerich goes on to say:

On the next page the results of the interrogation are recorded. In Italian are Galileo's words: 'I do not hold and have not held this opinion of Copernicus since the command was intimated to me that I must abandon it.' Then he was again told to speak the truth under the threat of torture. He responded: 'I am here to submit, and I have not held this opinion since the decision was pronounced, as I have stated.' Finally, there is a notation that nothing further could be done, and this time the document is properly signed in Galileo's hand. Galileo was sent back to his house at Arcetri, outside Florence, where he remained under house arrest until his death in 1642.

(Emphasis adde... (read more)

6Jayson_Virissimo
I got burned during a debate because I trusted the history from my physics textbook. After having read several books on the history of science (rather than summaries inside larger works) I am convinced that the Dark Arts on on full display even in natural science coursework.
5Tyrrell_McAllister
A gun can be used to commit a crime even if it isn't fired.
3steven0461
"Torture" here is analogous to "shooting", not "crime".
5Tyrrell_McAllister
I was analogizing "torture" with "gun", not "crime" or "shooting". Torture was a tool that the church had on hand and was prepared to use, and Galileo's knowledge of their threat to use torture was what led him to recant. (It was the forcing of his recanting that was the "crime" in my analogy.) It might be more precise to say that what the church had on hand was an institutionalized practice of torture, but using "torture" to refer to the practice (rather than a particular act) seems within the bounds of accuracy in poetry.
3Emile
That's a bit contrived - imagine if a presidential candidate mentions how his will was broken by torture in Vietnam, and afterward it's revealed that all that happened was that he was told he might be tortured, so he spilled the beans immediately. I wouldn't expect his poll numbers to go up.

imagine if a presidential candidate mentions how his will was broken by torture in Vietnam, and afterward it's revealed that all that happened was that he was told he might be tortured, so he spilled the beans immediately. I wouldn't expect his poll numbers to go up.

I would still say that torture was used to break his will. To say this would be accurate, if not precise (because I'm not specifying whether I mean a particular act or an institutionalized practice). Whether his will proved too easy to break to satisfy the electorate is another matter.

0[anonymous]
What about a shooting? Can a gun be used to commit a shooting even if it isn't fired?
9magfrump
This was especially exciting due to my newfound knowledge that ballad meter can be sung to the tune of the Gilligan's Island theme.
5Mass_Driver
Who is Catherine Faber? Has she made anything public about herself other than this wonderful poem? Google and Wikipedia were not immediately helpful.
7Emile
From her website: From her bio:
0Tiiba
I want to upvote this twice.
4byrnema
This comment being upvoted +21 doesn't fit my model of LessWrong voting, because it personifies the natural world with a God-concept, even if it is advocating for science and evolution. Am I missing something?
7Perplexed
So should every every metaphor be voted down? Or just personifying metaphors? Or just metaphors mentioning deities? I downvoted it because it perpetuated the myth that Galileo was tortured. Plus, God knows, the poetry was pretty awful.
8Risto_Saarelma
I figure this particular one strikes some as a bit iffy since the metaphor is so close to the salient metaphor the actual creationists are using and treating as a non-metaphor. Metaphors, like "God wrote life", closely associated with unsympathetic real-world groups tend to carry a bit extra baggage. The matter is of course confused further by the original context where this was written as a response to creationists.
0NancyLebovitz
What details have you got about Galileo? I'd heard that he was shown the instruments of torture, and recanted at that point.
1Perplexed
Well, there is some dispute whether he was "shown the instruments". A historian named Gingerich apparently argues that the showing never took place. But, in any case, threat of torture is not torture - or at least it is not what comes to mind when the myth of torture is repeated. The myth is a falsehood, which, if repeated by someone who knows better, is usually referred to as a "lie".
1gwern
Sounds like mock executions - they're not actually being executed...
-1Tiiba
"Plus, God knows, the poetry was pretty awful." I could agree with you more. But I won't.
6Unnamed
It beautifully promotes Joy in the Merely Real, and strongly encourages the pursuit of knowledge.
6Alicorn
It's good art advocating for science.
5jaimeastorga2000
I suspect it may be something similar to what NihilCredo said; rationalist quotes from theist sources are just so much fun.
0[anonymous]
That's actually pretty good, aesthetically. Thanks!
-6Apprentice
[-]anonym220

If you haven’t found something strange during the day, it hasn’t been much of a day.

John Archibald Wheeler

[-][anonymous]200

Getting caught up in style and throwing away victory is something for the lower ranks to do. Captains can't even think about doing such a carefree thing. Don't try to be a good guy. It doesn't matter who owes who. From the instant they enter into a war, both sides are evil.

Related to: Politics, Protection

A horse that can count to ten is a remarkable horse, not a remarkable mathematician.
--Samuel Johnson

[-]aausch180

"The best thing for being sad," replied Merlin, beginning to puff and blow, "is to learn something. That's the only thing that never fails. You may grow old and trembling in your anatomies, you may lie awake at night listening to the disorder of your veins, you may miss your only love, you may see the world about you devastated by evil lunatics, or know your honour trampled in the sewers of baser minds. There is only one thing for it then — to learn. Learn why the world wags and what wags it. That is the only thing which the mind can never exhaust, never alienate, never be tortured by, never fear or distrust, and never dream of regretting. Learning is the only thing for you. Look what a lot of things there are to learn."

— T.H. White (The Once and Future King)

4soreff
There are exceptions... When a child first learns that he or she is mortal, I doubt that that is a happy day for him or her. Truths are valuable, but some are rather bitter.
0avalot
Yes, and I think this is the one big crucial exception... That is the one bit of knowledge that is truly evil. The one datum that is unbearable torture on the mind. In that sense, one could define an adult mind as a normal (child) mind poisoned by the knowledge-of-death toxin. The older the mind, the more extensive the damage. Most of us might see it more as a catalyst than a poison, but I think that's insanity justifying itself. We're all walking around in a state of deep existential panic, and that makes us weaker than children.
3rwallace
Well, it's not the knowledge of death that's evil, it's the actual phenomenon -- there's not much point blaming the messenger for the bad news. Especially not now we're at the stage where we're beginning to have a chance to do something about it.
2AlanCrowe
Ernest Becker agrees with you, but I always read the one star reviews first. For myself, I've lost touch with Becker's ontology. I'm reduced to making the lame suggestion of playing Go in tournaments in order to practice managing a limited stock of time, such as 70 years.

For to be possessed of a vigorous mind is not enough; the prime requisite is rightly to apply it. The greatest minds, as they are capable of the highest excellences, are open likewise to the greatest aberrations; and those who travel very slowly may yet make far greater progress, provided they keep always to the straight road, than those who, while they run, forsake it.

~René Descartes, Discourse on the Method

It is a profound and necessary truth that the deep things in science are not found because they are useful, they are found because it was possible to find them. -J. Robert Oppenheimer.

3bentarm
There are two quite different interpretations of this quote: it either says something about scientists, or something about scientific truths, and I'm not sure which is the intention. The two messages I see are: 1. Scientists just enjoy seeking truths, you don't need to give them the incentive of practical applications in order for them to do science, so any truths that can be discovered will be, regardless of their usefulness. 2. There are an awful lot of true things. The ones that we know might not be the most useful, but they are the ones that happen to lie in the (extremely small?) subset of true things that humans are capable of understanding. To an extent, I guess both of these are true... which one was Oppenheimer aiming at?
2Perplexed
Quibble: Two things you might have missed: * Oppenheimer was talking about "deep things in science", not about "truths." * He said "possible to find them", not "possible to understand them".
-1stochastic
There are an awful lot of true things. I think that many of the things that are commonly regarded as being "true" are socially constructed fictions, biases and fallacies. Moreover science can never attain absolute truth it can only strive for it.
5orthonormal
Hi stochastic, and welcome to Less Wrong! This is actually a really important topic. I agree that there are a lot of cultural and normative claims that don't deserve to be called "true" or "false", despite their common usage as such. I'd be cautious of using the phrase "absolute truth", since it conjures up false expectations compared to the actual process of increasing confidence in models of the world. Really relevant: The Simple Truth P.S. Introduce yourself on the welcome page when you have a moment!
3Emile
The quote syntax is > quote goes here Which becomes
0wedrifid
Profound, necessary and optimistic. :)

David Hume was right to predict that superstition would survive for hundreds of years after his death, but how could he have anticipated that his own work would inspire Kant to invent a whole new package of superstitions? Or that the incoherent system of Marx would move vast populations to engineer their own ruin? Or that the infantile rantings of the author of Mein Kampf would be capable of bringing the whole world to war?

Perhaps we will one day succeed in immunizing our societies against such bouts of collective idiocy by establishing a social contract in which each child is systematically instructed in Humean skepticism. Such a new Emile would learn about the psychological weaknesses to which Homo sapiens is prey, and so would understand the wisdom of treating all authorities - political leaders and social role-models, academics and teachers, philosophers and prophets, poets and pop stars - as so many potential rogues and knoves, each out to exploit the universal human hunger for social status. He would therefore appreciate the necessity of doing all of his own thinking for himself. He would understand why and when to trust his neighbors. Above all, he would waste no time yearning for utopias that are incompatible with human nature.

-- Ken Binmore, in Natural Justice, p56

7MichaelVassar
Science works by scientists not doing all their thinking for themselves. That's also how it fails. Getting the balance right may be hard, but no-one has really tried very hard, so it may not be. Trying to do that is largely what I see SIAI as being about.
6Perplexed
Hmmm. A mathematician learning a new field thinks for himself, up to a point. Oh, he gets his ideas, theorems, and even proofs from the book, but he is supposed to verify the thinking for himself. The same kind of thing applies to scientists. They get ideas, formulas, and even empirical data from other scientists, but they are supposed to verify the inferences and even some of the derivations themselves. At least in their own field. A neuroscientist using FMRI doesn't need to know the fine points of the portions of QED dealing with particle spins in a varying magnetic field. Nor the computer science involved in the image processing. But he does appreciate that these tools, whether he understands them in detail himself or not, are not based on tradition or authority, but instead draw their legitimacy from the work of his colleagues in those fields who definitely do think for themselves. If the balance you seek to strike is the balance that lets you distinguish path-breaking innovation from crackpottery, I would suggest this: It is ok to try doing something that the experts think is impossible if you really understand why they are so pessimistic and you think you might understand why they are wrong.
0simplyeric
I'm not sure that's true. The issue isn't what a person "thinks"...it's what a person ultimately concludes. A scientist must think for itself in order to hypothesize, no? I think science goes wrong when scientists conclude for themselves, in the face of the actul facts on the matter. I think what is being referenced above is how to separate information from who said it, and how.
2PhilGoetz
I like the sentiment, but - instructed in Humean skepticism? Isn't that going overboard in the opposite direction?
8Perplexed
Binmore is on something of a "Hume is God, Kant is Satan" kick in this book. Another quote I like deals with Binmore's efforts to comprehend the "categorical imperative": I share much of Binmore's enthusiasm for Hume. I don't think that rationalists have much reason to dislike Hume's skepticism. Hume was a practical man, and his famous argument against induction is far from a counsel of epistemological despair. As for instructing the young to be skeptical of gods - well it may violate the US Constitution, but then so does gun control. ;) Nonetheless, I suspect that many people here would not care much for this particular quote in its full context - starting a couple paragraphs before my quote and continuing a paragraph further.
[-]XiXiDu150

This is a bit long for a rationality quote and isn't really a quote but short enough and worth the read: The most poetic and convincing argument for striving for posthumanity (via aleph.se).

4Pavitra
That's kind of depressing.
[-]xamdam150

We're all puppets, Laurie. I'm just a puppet who can see the strings.

Dr. Manhattan (Watchmen)

It is still an unending source of surprise for me how a few scribbles on a blackboard or on a piece of paper can change the course of human affairs. -Stanislaw Ulam

0wedrifid
Can they really? I have my doubts. Most of those scribbles on a blackboard were either an inevitable result of outside forces or would have been made on a different blackboard had they they not been made there. (Although to be fair the butterfly and mere chance will play their part at least some of the time.)
6Perplexed
Scribbles on maps, particularly in 1815 and 1919, had some largish effects.

In 1923, England and France divided between them the previously Turkish territories of what are modern Syria, Lebanon and Israel/Palestine. They drew a pencil line on a map to mark the treaty border.

It turned out that the thickness of the pencil line itself was several hundred meters on the ground. In 1964, Israel fought a battle with Syria over that land.

People were killed because someone neglected to sharpen their pencil. That's "scribbles on a piece of paper" for you.

Ref: a book found by Google. I originally learned about this from an Israeli plaque at the Dan River preserve near the border.

7wedrifid
I suppose it would be in bad taste to find that rather amusing. Or at least to admit it.
6James_K
In circumstances like that I find I have to laugh, if only to keep from weeping.
1Drawbacks
"The 350-mile detour in the Trans-Siberian Railway was caused by the Tsar, who drew the proposed route using a ruler with a notch in it." -- Not 1982
0Pfft
What's the source for this? Googling "Not 1982" is not helpful... I did find the following amusing quote though: "The Trans-Siberian Railway". In The Living Age, seventh series volume five, 1899
1Manfred
http://en.wikipedia.org/wiki/Not_the_Nine_O%27Clock_News#Books_and_miscellaneous My google-fu is strong-ish. Still, not a particularly reliable source.
0gwern
I wonder if Nicholas was acting in the same spirit as King Canute and likewise has been subsequently misinterpreted. (I've seen the Canute story mentioned as an example of being power-mad.) Nicholas's intention could have been something like 'Gentlemen, you were chosen for your competence in engineering and expertise in dealing with such details; I have made my general wish known to you; kindly implement it and do not bother me with what is your job.'
1gwern
The partitions of Korea and Vietnam are some more recent examples; nor have we seen the last of the largish effects of the former.
1PhilGoetz
He could have also been thinking about the Declaration of Independence, the Declaration of the Rights of Man, and various other documents. (I'd list the Magna Carta, but it didn't really have the effect it's credited with. It was a few lines in a larger document that was more concerned with the hunting privileges of nobles than with the rights of man, and that was nullified before the year was out.)
-1MichaelGR
I think he had things like the development of physics in the 20th century that led to the creation of the A and H bombs. I got the quote from Richard Rhodes history of the making of the atomic bomb. It doesn't matter exactly which blackboard or wrote wrote what, in the end, a bunch of people making calculations and experiments changed the course of human affairs pretty significantly.

If, instead of welcoming inquiry and criticism, the admirers of a great author accept his writings as authoritative, both in their excellences and in their defects, the most serious injury is done to truth. In matters of philosophy and science, authority has ever been the great opponent of truth. A despotic calm is usually the triumph of error. In the republic of the sciences, sedition and even anarchy are beneficial in the long run to the greatest happiness of the greatest number.

William Stanley Jevons, Theory of Political Economy, 1871: p.275-6

“But for that matter, how do you explain the fact that the statues of Easter Island are megaliths exactly like the Celtic ones? Or that a Polynesian god called Ya is clearly the Yod of the Jews, as is the ancient Hungarian Io-v’, the great and good god? Or that an ancient Mexican manuscript shows the Earth as a square surrounded by sea, and in its center is a pyramid that has on its base the inscription Aztlan, which is close to Atlas and Atlantis? Why are pyramids found on both sides of the Atlantic?”

“Because it’s easier to build pyramids than spheres. Because the wind produces dunes in the shape of pyramids and not in the shape of the Parthenon.”

“I hate the spirit of the Enlightenment.”

Umberto Eco, Foucault's Pendulum

Many a man has cherished for years as his hobby some vague shadow of an idea, too meaningless to be positively false; he has, nevertheless, passionately loved it, has made it his companion by day and by night, and has given to it his strength and his life, leaving all other occupations for its sake, and in short has lived with it and for it, until it has become, as it were, flesh of his flesh and bone of his bone; and then he has waked up some bright morning to find it gone, clean vanished away like the beautiful Melusina of the fable, and the essence of his life gone with it. I have myself known such a man; and who can tell how many histories of circle-squarers, metaphysicians, astrologers, and what not, may not be told in the old German story?

Charles Sanders Peirce

1Perplexed
Ouch. But that part about "the essence of his life gone with it" is an exaggeration - or at least, only a temporary loss. There are plenty of vague shadows of ideas out there to be loved and cherished.

A book is a machine to think with.

I. A. Richards, "Principles of Literary Criticism"

3satt
Reminiscent of Umberto Eco describing the novel as "a machine for generating interpretations".
1sketerpot
Take that idea far enough and you get something like Haibane Renmei, where there is no official interpretation -- everybody has to generate their own idea of what the show's premise is. This was frustrating the first time I watched it, since I didn't know that there wasn't going to be an explanation for everything. The second time, though, I absolutely loved it.

If it's a stupid idea and it works, then it isn't stupid.

-- French Ninja, Freefall

Puts me in mind of "Rationalists should win".

1Document
-- benelliott (edited to attribute)

If you can't tell whose side someone is on, they are not on yours. -Warren E. Buffett

4xamdam
If after 1/2 hr of poker you can't tell who's the patsy, it's you. - Charles T. Munger
0Document
Sounds like one for the quotes page for "Default to Good" at TV Tropes. (Link omitted due to time hazard.)
-1Dre
Wouldn't this be a problem for tit for tat players going up against other tit for tat players (but not knowing the strategy of their opponent)?
0orthonormal
Only if it's common knowledge that both players are human. ETA: Since I got downvoted, maybe I wasn't being clear. I think that the Warren Buffett quote applies to human psychology more than to game theory in general. If outright deception were easy, it would probably become a good strategy to keep your allies in some doubt about your intentions, as a bargaining chip. But we humans don't seem to be good at pulling that off, and so ambivalence is a strong signal of opposition.
1Perplexed
Now that you have clarified, I wish I could downvote a second time. Tit-for-tat is a good strategy in the iterated prisoner's dilemma regardless of whether the players are human and regardless of whether the other player is "on your side". In fact, it is pretty much taken for granted that there are no sides in the PD. Dre was downvoted by me for a complete misunderstanding of how Tit-for-tat relates to "sides". You were downvoted for continuing the confusion.
3orthonormal
Oh, you're right- my response would have made sense talking about players in a one-shot PD with communication beforehand, but it's a non sequitur to Dre's mistaken comment. Don't know how I missed that.
0Perplexed
Upvoted, but even with communication beforehand, the rational move in a one-shot PD is to defect. Unless there is some way to make binding commitments, or unless there is some kind of weird acausal influence connecting the players. Regardless of whether the other player is human and rational, or silicon and dumb as a rock.
0Vladimir_Nesov
Taboo "rational". Acausal control is not something additional, it's structure that already exists in a system if you know where to look for it. And typically, it's everywhere, to some extent.
1shokwave
Highest-scoring move, adjective applied to the course that maximises fulfillment of desires. The best move in a one-shot PD is to defect against a cooperator. With no communication or precommitment, and with the knowledge that it is a one-shot PD, the overwhelming outcome is both defect. Adding communication to the mix creates a non-zero chance you can convince your opponent to cooperate - which increases the utility of defecting.
0Vladimir_Nesov
There is a question of what will actually happen, but also more relevant questions of what will happen if you do X, for various values of X. If you convince the opponent to cooperate, it's one thing, not related to the case of convincing your opponent to cooperate if you cooperate.
0shokwave
Determine what kinds of control influence your opponent, appear to also be influenced by the same, and then defect when they think you are forced into cooperating because they are forced into cooperating? Is that a legitimate strategy, or am I misunderstanding what you mean by convincing your opponent to cooperate if you cooperate?
1Vladimir_Nesov
Couldn't parse.
1Vladimir_Nesov
It's not in general possible to predict what you'll actually do, since if it were possible, you could take such predictions into consideration in deciding what to do, in particular you could decide differently as a result, invalidating the "prediction". Similarly, it's not in general possible to predict what will actually happen, without assuming what you'll decide first. It's better to ask, what is likely to happen if you decide X, than to ask just what is likely to happen. It's more useful too, since it gives you information about (acausal) consequences of your actions that can be used as basis for making decisions. In the case of Prisoner's Dilemma, it's not very helpful to ask, what will your opponent do. What your opponent will do generally depends on what you'll do, and assuming that it doesn't is a mistake that leads to the classical conclusion that defecting is always the better option (falsified by the case of identical players that always make the same decision, with cooperation the better one). If you ask instead, what will your opponent do (1) if you cooperate, and (2) if you defect, that can sometimes give you interesting answers, such that cooperating suddenly becomes the better option. When you talk to the opponent with the intention of "convincing" them, again you are affecting both predictions about what they'll do, on both sides of your possible decision, and not just the monolithic prediction of what they'll do unconditionally. In particular, you might want to influence the probability of your opponent cooperating with you if you cooperate, without similarly affecting the probability of your opponent cooperating with you if you defect. If you affect both probabilities in the same way, then you are correct, such influence makes the decision of defecting more profitable than before. But if you affect these probabilities to a different degree, then it might turn out that the opposite is true, that the influence in question makes cooperating more pro
2shokwave
Ah, I see! I have been butting my head against various ideas that lead to cooperating in one-shot PDs and the like and not making any progress, it was because while I had the idea of splitting my actions into groups conditional on the opponent's action, I didn't have the concept of doing the same for my opponent. With that in mind, I can no longer parse my previous comment either. I think I meant that I would increase their probability of cooperating if I cooperated, and have them increase my probability of cooperating if they cooperated (thus decreasing both of our probabilities of defecting if the other cooperates), and then when the probabilities have moved far enough to tell us both to cooperate, I would defect, knowing that I would score a defect-against-cooperate. But yeah, it doesn't make any sense at all, because the probabilities tell us both to cooperate. Thanks for taking the time to explain this concept to me.
0[anonymous]
(Note that probability of you making a given decision is not knowable, when you are considering it yourself while allowing this consideration to influence the decision.)
0orthonormal
Perplexed, have you come across the decision theory posts here yet? You'll find them pretty interesting, I think. LW Wiki for the Prisoner's Dilemma LW Wiki for timeless decision theory (start with the posts- Eliezer's PDF is very long and spends more time justifying than explaining). Essentially, this may be beyond the level of humans to implement, but there are decision theories for an AI which do strictly better than the usual causal decision theory, without being exploitable. Two of these would cooperate with each other on the PD, given a chance to communicate beforehand.
1Perplexed
Yes, I have read them, and commented on them. Negatively, for the most part. If any of these ideas are ever published in the peer reviewed literature, I will be both surprised and eager to read more. I think that you may have been misled by marketing hype. Even the proponents of those theories admit that they do not do strictly better (or at least as good) on all problems. They do better on some problems, and worse on others. Furthermore, sharing source code only provides a guarantee that the observed source is current if that source code cannot be changed. In other words, an AI that uses this technique to achieve commitment has also forsaken (at least temporarily) the option of learning from experience. I am intrigued by the analogy between these acausal decision theories and the analysis of Hamilton's rule in evolutionary biology. Nevertheless, I am completely mystified as to the motivation that the SIAI has for pursuing these topics. If the objective is to get two AIs to cooperate with each other there are a plethora of ways to do that already well known in the game theory canon. An exchange of hostages, for example, is one obvious way to achieve mutual enforceable commitment. Why is there this fascination with the bizarre here? Why so little reference to the existing literature?
2WrongBot
So far as I understand the situation, the SIAI is working on decision theory because they want to be able to create an AI that can be guaranteed not to modify its own decision function. There are circumstances where CDT agents will self-modify to use a different decision theory (e.g. Parfit's Hitchhiker). If this happens (they believe), it will present a risk of goal-distortion, which is unFriendly. Put another way: the objective isn't to get two AIs to cooperate, the objective is to make it so that an AI won't need to alter its decision function in order to cooperate with another AI. (Or any other theoretical bargaining partner.) Does that make any sense? As a disclaimer, I definitely do not understand the issues here as well as the SIAI folks working on them.
2orthonormal
I don't think that's quite right- a sufficiently smart Friendly CDT agent could self-modify into a TDT (or higher decision theory) agent without compromising Friendliness (albeit with the ugly hack of remaining CDT with respect to consequences that happened causally before the change). As far as I understand SIAI, the idea is that decision theory is the basis of their proposed AI architecture, and they think it's more promising than other AGI approaches and better suited to Friendliness content.
0Perplexed
That sounds intriguing also. Again, a reference to something written by someone who understands it better might be helpful so as to make some sense of it.
1cousin_it
Maybe it would be helpful to you to think of self-modifications and alternative decision theories as unrestricted precommitment. If you had the ability to irrevocably precommit to following any decision rule in the future, which rule would you choose? Surely it wouldn't be pure CDT, because you can tractably identify situations where CDT loses.
0Perplexed
"Tractably" is a word that I find a bit unexpected in this context. What do you mean by it? "Situations where CDT loses." Are we talking about real-world-ish situations here? Situations in which causality applies? Situations in which the agents are free rather than being agents whose decisions have already been made for them by a programmer at some time in the past? What kind of situations do you have in mind? And what do you mean by "loses"? Loses to who or what? Loses to agents that can foresee their opponent's plays? Agents that have access to information channels not available to the CDT agent? Just what information channels are allowed? Why those, and not others? ETA: And that "Surely it wouldn't be CDT ... because you can identify ..." construction simply begs for completion with "Surely it would be ... because you can't identify ...". Do you have a candidate? Do you have a proof of "you can't identify situations where it loses". If not, what grounds do you have for criticizing?
0[anonymous]
CDT still loses to TDT in Newcomb's problem if Omega has can predict your actions with better than 50.05% chances. You can't get out of this by claiming that Omega has access to unrealistic information channels, because these chances seem fairly realistic to me.
0WrongBot
Free from what? Causality? This sounds distressingly like you are relying on some notion of "free will". (Apologies if I'm misreading you.)
0Perplexed
I am relying on a notion of free will. I understand that every normative decision theory adopts the assumption (convenient fiction if you prefer) that the agent being advised is acting of "his own free will". Otherwise, why bother advising? Being a compatibilist, as I understand Holy Scripture (i.e. The Sequences) instructs me to be, I see no incompatibility between this "fiction" of free will and the similar fiction of determinism. They model reality at different levels. For certain purposes, it is convenient to model myself and other "free agents" as totally free in our decisions, but not completely free in carrying out those decisions. For example, my free will ego may decide to quit smoking, but my determined id has some probability of overruling that decision.
2WrongBot
Why the distinction between agents which are free and agents which have had their decisions made for them by a programmer, then? Are you talking about cases in which specific circumstances have hard-coded behavioral responses? Every decision every agent makes is ultimately made for it by the agent's programmer; I suppose I'm wondering where you draw the line. As a side note, I feel very uncomfortable seeing the sequences referred to as inviolable scripture, even in jest. In my head, it just screams "oh my god how could anyone ever be doing it this wrong arghhhhhh." I'm still trying to figure out what I think of that reaction, and do not mention it as a criticism. I think.
1Perplexed
I make the distinction because the distinction is important. The programmer makes decisions at one point in time, with his own goals and/or utility functions, and his own knowledge of the world. The agent makes decisions at a different point in time, based on different values and different knowledge of the world. A decision theory which advises the programmer is not superior to a decision theory which advises the agent. Those two decision theories are playing different games.
1nshepperd
"Totally free" sounds like too free. You're not free to actually decide at time T to "decide X at time T+1" and then actually decide Y at time T+1, since that is against the laws of physics. It's my understanding that what goes through your head when you actually decide X at time T+1 is (approximately) what we call TDT. Or you can stick to CDT and not be able to make decisions for your future self.
0Perplexed
I upvoted this because it seems to contain a grain of truth, but I'm nervous that someone before me had downvoted it. I don't know whether that was because it actually is just completely wrong about what TDT is all about, or because you went a bit over the top with "against the laws of physics".
0cousin_it
Situations where CDT loses are precisely those situations where credible precommitment helps you, and inability to credibly precommit hurts you. There's no shortage of those in game theory.
1Perplexed
Ok, those are indeed a reasonable class of decisions to consider. Now, you say that CDT loses. Ok, loses to what? And presumably you don't mean loses to opponents of your preferred decision theory. You mean loses in the sense of doing less well in the same situation. Now, presumably that means that both CDT and your candidate are playing against the same game opponent, right? I think you see where I am going here, though I can spell it out if you wish. In claiming the superiority of the other decision theory you are changing the game in an unfair way by opening a communication channel that didn't exist in the original game statement and which CDT has no way to make use of.
1cousin_it
Well, yeah, kind of, that's one way to look at it. Reformulate the question like this: what would CDT do if that communication channel were available? What general precommitment for future situations would CDT adopt and publish? That's the question TDT people are trying to solve.
1Perplexed
The simplest answer that moves this conversation forward would be "It would pretend to be a TDT agent that keeps its commitments, whenever that act of deception is beneficial to it. It would keep accurate statistics on how often agents claiming to be TDT agents actually are TDT agents, and adjust its priors accordingly." Now it is your turn to explain why this strategy violates the rules, whereas your invention of a deception-free channel did not.
0orthonormal
I'm going to have to refer you to Eliezer's TDT document for that. (If you're OK with starting in medias res, the first mention of this is on pages 22-23, though there it's just specialized to Newcomb's Dilemmas; see pages 50-52 for an example of the limits of this hack. Elsewhere he's argued for the more general nature of the hack.)
2Perplexed
Ok thanks. I'm coming to realize just how much of this stuff derives from Eliezer's insistance on reflective consistency of a decision theory. Given any decision theory, Eliezer will find an Omega to overthrow it. But doesn't a diagonal argument show that no decision theory can be reflectively consistent over all test data presented by a malicious Omega? Just as there is no enumeration of the reals, isn't there a game which can make any specified rational agent regret its rationality? Omega holds all the cards. He can always make you regret your choice of decision theory.
3jimrandomh
No. We can ensure that no such problem exists if we assume that (1) only the output decisions are used, not any internals; and (2) every decision is made with access to the full problem statement.
1bentarm
I'm not entirely sure what "every decision is made with full access to the problem statement means", but I can't see how it can possibly get around the diagonalisation argument. Basically, Omega just says "I simulated your decision on problem A, on which your algorithm outputs something different from algorithm X, and give you a shiny black ferrari iff you made the same decision as algorithm X" As cousin_it pointed out last time I brought this up, Caspian made this argument in response to the very first post on the Counterfactual Mugging. I've yet to see anyone point out a flaw in it as an existence proof. As far as I can see the only premise needed for this diagonalisation to work is that your decision theory doesn't agree with algorithm X on all possible decisions, so just make algorithm X "whatever happens, recite the Bible backwards 17 times".
4jimrandomh
In that case, your answer to problem A is being used in a context other than problem A. That other context is the real problem statement, and you didn't have it when you chose your answer to A, so it violates the assumption.
4Sniffnoy
Yeah, that definitely violates the "every decision is made with full access to the problem statement" condition. The outcome depends on your decision on problem A, but when making your decision on problem A you have no knowledge that your decision will also be used for this purpose.
1bentarm
I don't see how this is useful. Let's take a concrete example, let's have decision problem A, Omega offers you the choice of $1,000,000, or being slapped in the face with a wet fish. Which would you like your decision theory to choose? Now, No-mega can simulate you, say, 10 minutes before you find out who he is, and give you 3^^^3 utilons iff you chose the fish-slapping. So your algorithm has to include some sort of prior on the existence of "fish-slapping"-No-megas. My algorithm "always get slapped in the face with a wet fish where that's an option", does better than any sensible algorithm on this particular problem, and I don't see how this problem is noticeably less realistic than any others. In other words, I guess I might be willing to believe that you can get around diagonalisation by posing some stringent limits on what sort of all-powerful Omegas you allow (can anyone point me to a proof of that?) but I don't see how it's interesting.
4jimrandomh
Actually, no, the probability of fish-slapping No-megas is part of the input given to the decision theory, not part of the decision theory itself. And since every decision theory problem statement comes with an implied claim that it contains all relevant information (a completely unavoidable simplifying assumption), this probability is set to zero. Decision theory is not about determining what sorts of problems are plausible, it's about getting from a fully-specified problem description to an optimal answer. Your diagonalization argument requires that the problem not be fully specified in the first place.
0NihilCredo
This is a no-choice scenario. If you say that the Bible-reciter is the one that will "win" here, you are using the verb "to win" with a different meaning from the one used when we say that a particular agent "wins" by making the choice that leads to the best outcome.
2NihilCredo
With the strong disclaimer that I have no background in decision theory beyond casually reading LW... I don't think so. The point of simulation (Omega) problems, to me, doesn't seem to be to judo your intelligence against yourself; rather, it is to "throw your DT off the scent", building weird connections between events (weird, but still vaguely possible, at least for AIs), that a particular DT isn't capable of spotting and taking into account. My human, real-life decision theory can be summarised as "look at as many possible end-result worlds as I can, and at what actions will bring them into being; evaluate how much I like each of them; then figure out which actions are most efficient at leading to the best worlds". But that doesn't exactly fly when you're programming a computer, you need something that can be fully formalised, and that is where those strange Omega scenarios are useful, because your code must get it right "on autopilot", it cannot improvise a smarter approach on the spot - the formula is on paper, and if it can't solve a given problem, but another one can, it means that there is room for improvement. In short, DT problems are just clever software debugging.
2Perplexed
I agreed with everything you said after "I don't think so". So I am left confused as to why you don't think so. You analogize DT problems as test data used to determine whether we should accept or reject a decision theory. I am claiming that our requirements (i.e. "reflective consistency") are so unrealistic that we will always be able to find test data forcing us to reject. Why do you not think so?
0NihilCredo
Because I suspect that there are only so many functionally different types of connections between events (at the very least, I see no hint that there must be infinitely many) and once you've found them all you will have the possibility of writing a DT that can't be led to corner itself into suboptimal outcomes due to blind spots.
2Perplexed
Am I correct in interpreting this as "infinitely many of them"? If so, I am curious as to what you mean by "functionally different types of connections between events". Could you provide an example of some "types of connections between events"? Functionally different ones to be sure. Presumably, the relevance must be your belief that decision theories differ in just how many of these different kinds of connections they handle correctly. Could you illustrate this by pointing out how the decision theory of your choice handles some types of connections, and why you have confidence that it does so correctly?
0NihilCredo
Oops, yes. Fixed. CDT can 'see' the classical, everyday causal connections that are marked in formulas with the symbol ">" (and I'd have to spend several hours reading at least the Stanford Encyclopaedia before I could give you a confident definition of that), but it cannot 'see' the connection in Newcomb's problem between the agent's choice of boxes and the content of the opaque box (sometimes called 'retrocausality'). I don't have a favourite formal decision theory, because I am not sufficiently familiar with the underlying math and with the literature of discriminating scenarios to pick a horse. If you're talking about the human decision "theory" of mine I described above, it doesn't explicitly do that; the key hand-waving passage is "figure out which actions are most efficient at leading to the best worlds", meaning I'll use whatever knowledge I currently possess to estimate how big is the set of Everett branches where I do X and get A, compared to the set of those where I do X and get B. (For example, six months ago I hadn't heard of the concept of acausal connections and didn't account for them at all while plotting the likelihoods of possible futures, whereas now I do - at least technically; in practice, I think that between human agents they are a negligible factor. For another example, suppose that some years from now I became convinced that the complexity of human minds, and the variability between different ones, were much greater than I previously thought; then, given the formulation of Newcomb's problem where Omega isn't explicitly defined as a perfect simulator and all we know is that it has had a 100% success rate so far, I would suitably increase my estimation of the chances of Omega screwing up and making two-boxing profitable.)
1Perplexed
Ok, so if I understand you, there are only some finite number of valid kinds of connections between events and when we have all of them incorporated - when our decision theory can "see" each of them - we are then all done. We have the final, perfect decision theory (FPDT). But what do you do then when someone - call him Yuri Geller - comes along and points out that we left out one important kind of connection: the "superspooky" connection. And then he provides some very impressive statistical evidence that this connection exists and sets up games in front of large (paying) audiences in which FPDT agents fail to WIN. He then proclaims the need for SSPDT. Or, if you don't buy that, maybe you will prefer this one. Yuri Geller doesn't really exist. He is a thought experiment. Still the existence of even the possibility of superspooky connections proves that they really do exist and hence that we need to have SADT - Saint Anselm's Decision Theory. Ok, I've allowed my sarcasm to get the better of me. But the question remains - how are you ever going to know that you have covered all possible kinds of connections between events?
0NihilCredo
You can't, I guess. Within an established mathematical model, it may be possible to prove that a list of possible configurations of event pairs {A, B} is exhaustive. But the model may always prove in need of expansion or refinement - whether because some element gets understood and modellised at a deeper level (eg the nature of 'free' will) or, more worryingly, because of paradigm shifts about physical reality (eg turns out we can time travel).
0Vladimir_Nesov
Decision theories should usually be seen as normative, not descriptive. How "realistic" something is, is not very important, especially for thought experiments. Decision theory cashes out where you find a situation that can indeed be analyzed with it, and where you'll secure a better outcome by following theory's advice. For example, noticing acausal control has advantages in many real-world situations (Parfit's Hitchhiker variants). Eliezer's TDT paper discusses this towards the end of Part I.
0Perplexed
I believe you misinterpreted my "unrealistic requirements". A better choice of words would have been "unachievably stringent requirements". I wasn't complaining that Omega and the like are unrealistic. At least not here. The version I have of Eliezer's TDT paper doesn't have a "Part I". It is dated "September 2010 and has 112 pages. Is there a better version available? I don't understand your other comments. Or, perhaps more accurately, I don't understand what they were in response to.
0Vladimir_Nesov
"Part I" is chapters 1-9. (This concept is referred to in the paper itself.)
0Perplexed
Not to me. But a reference might repair that deficiency on my part.
0WrongBot
See Eliezer's posts on Newcomb's Problem and regret of rationality and TDT problems he can't solve. (Incidentally, I found those reference in about 30 seconds, starting from the LW Wiki page on Parfit's Hitchhiker.)
-3Perplexed
Ah! Thank you. I see now. The circumstance in which a CDT agent will self modify to use a different decision theory are that: * The agent was programmed by Eliezer Yudkowsky and hence is just looking for an excuse to self-modify. * The agent is provided with a prior leading it to be open to the possibility of omnicient, yet perverse agents bearing boxes full of money. * The agent is supplied with (presumably faked) empirical data leading it to believe that all such omniscient agents reward one-boxers. * Since the agent seeks reflective equilibrium (because programmed by aforesaid Yudkowsky), and since it knows that CDT requires two boxing, and since it has no reason to doubt that causality is important in this world, it makes exactly the change to its decision theory that seems appropriate. It continues to use CDT except on Newcomb problems, where it one boxes. That is, it self-modifies to use a different decision theory, which we can call CDTEONPWIOB. Well, ok, though I wouldn't have said that these are cases where CDT agents do something weird. These are cases where EYDT agents do something weird. I apologize if it seems that the target of my sarcasm is you WrongBot. It is not. EY has deluded himself into thinking that reflective consistency is some kind of gold standard of cognitive stability. And then he uses reflective consistency as a lever by which completely fictitious data can uproot the fundamental algorithms of rationality. Which would be fine, except that he has apparently convinced a lot of smart people here that he knows what he is talking about. Even though he has published nothing on the topic. Even though other smart people like Robin tell him that he is trying to solve an already solved problem. I would say more but ... This manuscript was cut off here, but interested readers are suggested to look at these sources for more discussion: Bibliography Gibbard, A., and Harper, W. L. (1978), "Counterfactuals and Two Kinds of Expected Utility",
3Sniffnoy
Reflective consistency is not a "gold standard". It is a basic requirement. It should be easy to come up with terrible, perverse decision theories that are reflectively consistent (EY does so, sort of, in his TDT outline, though it's not exactly serious / thorough). The point is not that reflective consistency is a sign you're on the right track, but that a lack of it is a sign that something is really wrong, that your decision theory is perverse. If using your decision theory causes you to abandon that same decision theory, it can't have been a very good decision theory. Consider it as being something like monotonicity in a voting system; it's a weak requirement for weeding out things that are clearly bad. (Well, perhaps not everyone would agree IRV is "clearly bad", but... it isn't even monotonic!) It just happens that in this case evidently nobody noticed before that this would be a good condition to satisfy and hence didn't try. :)
0timtyler
Am not sure that decision theory is an "already solved" problem. There's the issue of what happens when agents can self-modify - and so wirehead themselves. I am pretty sure that is an unresolved "grand challenge" problem.
0WrongBot
TDT gets better outcomes than CDT when faced with Newcomb's Problem, Parfit's Hitchhiker, and the True Prisoner's Dilemma. When does CDT outperform TDT? If the answer is "never", as it currently seems to be, why wouldn't a CDT agent self-modify to use TDT?
0Perplexed
Because it can't find a write-up that explains how to use it? Perhaps you can answer the questions that I asked here What play does TDT make in the game of Chicken? Can you point me to a description of TDT that would allow me to answer that question for myself?
1WrongBot
Suppose I'm an agent implementing TDT. My decision in Chicken depends on how much I know about my opponent. * If I know my opponent implements the same decision procedure I do (because I have access to its source code, say), and my opponent has this knowledge about me, I swerve. In this case, my opponent and I are in symmetrical positions and its choice is fully determined by mine; my choice is between payoffs of (0,0) and (-10,-10). * Else, I act identically to a CDT agent. As Eliezer says here, the one-sentence version of TDT is "Choose as though controlling the logical output of the abstract computation you implement, including the output of all other instantiations and simulations of that computation."
2Sniffnoy
I'm not sure this is right. Isn't there a correlated equilibrium that does better?
2WrongBot
I think we're looking at different payoff matrices. I was using the formulation of Chicken that rewards which doesn't have a correlated equilibrium that beats (C,C). Using the payoff matrix Perplexed posted here, there is indeed a correlated equilibrium, which I believe the TDT agents would arrive at (given a source of randomness). My bad for not specifying the exact game I was talking about.
0Sniffnoy
...and, this is what I get for not actually checking things before I post them.
0Perplexed
Two questions: 1. Why do you believe the TDT agents would find the correlated equilibrium? Your previous statement and Eliezer quote suggested that a pair of TDT agents would always play symmetrically in a symmetric game. No "spontaneous symmetry breaking". 2. Even without a shared random source, there is a Nash mixed equilibrium that is also better than symmetric cooperation. Do you believe TDT would play that if there were no shared random input?
0WrongBot
1. In a symmetric game, TDT agents choose symmetric strategies. Without a source of randomness, this entails playing symmetrically as well. 2. I'm not sure why you're talking about shared random input. If both agents get the same input, they can both be expected to treat it in the same way and make the same decision, regardless of the input's source. Each agent needs an independent source of randomness in order to play the mixed equilibrium; if my strategy is to play C 30% of the time, I need to know whether this iteration is part of that 30%, which I can't do deterministically because my opponent is simulating me.
0Sniffnoy
Yeah, I think any use of correlated equilibrium here is wrong - that requires a shared random source. I think in this case we just get symmetric strategies, i.e., it reduces to superrationality, where they each just get their own private random source.
0Perplexed
Sorry if this was unclear. It was a reference to the correlated pair of random variables used in a correlated equilibrium. I was saying that even without such a correlated pair, you may presume the availability of independent random variables which would allow a Nash equilibrium - still better than symmetric play in this game.
0Sniffnoy
Gah, wait. I feel dumb. Why would TDT find correlated equilibria? I think I had the "correlated equilibrium" concept confused. A correlated equilibrium would require a public random source, which two TDTers won't have.
3steven0461
Digits of pi are kind of like a public random source.
0Sniffnoy
Ignoring the whole pi-is-not-known-to-be-normal thing, how do you determine which digit of pi to use when you can't actually communicate and you have no idea how many digits of pi the other player may already know?
0steven0461
Same way you meet up in New York with someone you've never talked to: something like Schelling points. I'm not sure that answer works in practice.
1Perplexed
Thank you. I hope you realize that you have provided an example of a game in which CDT does better than TDT. For example, in the game with the payoff matrix shown below, there is a mixed strategy Nash equilibrium which is better than the symmetric cooperative result.
0WrongBot
Looks like we're talking about different versions of Chicken. Please see my reply to Sniffnoy.
0Perplexed
So TDT is different from CDT only in cases where the game is perfectly symmetric? If you are playing a game that is roughly the symmetric PD, except that one guy's payoffs are shifted by a tiny +epsilon, then they should both defect?
0WrongBot
TDT is different from CDT whenever one needs to consider the interaction of multiple decisions made using the same TDT-based decision procedure. This applies both to competitions between agents, as in the case of Chicken, and to cases where an agent needs to make credible precommitments, as in Newcomb's Problem. In the case of an almost-symmetric PD, the TDT agents should still cooperate. To change that, you'd have to make the PD asymmetrical enough that the agents were no longer evaluating their options in the same way. If a change is small enough that a CDT agent wouldn't change its strategy, TDT agents would also ignore it. This doesn't strike me as the world's greatest explanation, but I can't think of a better way to formulate it. Please let me know if there's something that's still unclear.
0Perplexed
This strikes me as a bit bizarre. You test whether a warped PD is still close enough to symmetric by asking whether a CDT agent still defects in order to decide whether a TDT agent should still cooperate? Are you sure you are not just making up these rules as you go? Much is unclear and very little seems to be coherently written down. What amazes me is that there is so much confidence given to something no one can explain clearly. So far, the only stable thing in your description of TDT is that it is better than CDT.
2JGWeissman
Do you have an example of a problem on which CDT or EDT does better than TDT?
4Perplexed
I have yet to see a description of TDT which allows me to calculate what TDT does on an arbitrary problem. But I do know that I have seen long lists from Eliezer of problems that TDT does not solve that he thinks it ought to be improved so as to solve.
0orthonormal
The world isn't sufficiently formalized for us to meet that standard for any decision theory (though we come closer with CDT and TDT than with EDT, in my opinion). However, cousin_it has a few recent posts on formalized situations where an agent of a more TDT (actually, UDT) type does strictly better than a CDT one in the same situation. I don't know of any formalization (or any fuzzy real-world situation) where the opposite is true.
0Perplexed
I apparently misled you by using that word "arbitrary". I'm not asking for solutions to soft problems that are difficult to formalize. Simply solutions to the standard kinds of games already formalized in game theory. For example, the game of Chicken). Can anyone point me to a description that tells me what play TDT would make in this game? Or what mixed strategy it would use? Both assuming and not assuming the reading of each other's code. ETA: Slightly more interesting than the payoff matrix shown in the wikipedia article is the case when the payoff for a win is 2 units, with a loss still costing only -1. This means that in the iterated version, the negotiated solution would be to alternate wins. But we are interested in the one-shot case. Can TDT find a correlated equilibrium? If not, which Nash equilibrium does it pick? Or does it always chicken out? Where can I learn this information?
0JGWeissman
Since CDT and EDT don't solve those problems either, all this justifies saying is that TDT does better on some problems, and the same on others, not "worse on others".
2timtyler
For every possible decision theory, there is a "nemesis" environment - where it does extremely badly. That is no-free-lunch fall out.
2JGWeissman
A "nemesis" environment that feeds misleading evidence to a decision theory's underlying epistimology does not indicate the sort of problem illustrated by an environment in which a decision theory does something stupid with true information.
0timtyler
What you asked for was a case where a decision theory did worse than its rivals. However, that seems pretty trivial if it behaves differently from them - you just consider an appropriate pathological environment set up to punish that decision theory.
0JGWeissman
Yes, in the context of Perplexed dismissing examples of TDT doing better than CDT because CDT was being stupid with true information.
0JoshuaZ
Not necessarily. Various decision theories can come into play here. It depends precisely on what you mean by the prisoner's paradox. If you are playing a true one shot where you have no information about the entity in question then that might be true. But if you are playing a true one shot where you each before making the decision have each player have access to the other player's source code then defecting may not be the best solution. Some of the decision theory posts have discussed this. (Note that knowing each others' source code is not nearly as strong an assumption as it might seem since one common idea in game theory is to look at what game theory occurs when people know when the other players know your strategy. (I'm oversimplifying some technical details here. I don't fully understand all the issues. I'm not a game theorist. Add any other relevant disclaimers.))
-2Perplexed
No one on this thread has mentioned a "prisoner's paradox". We have been discussing the Prisoner's Dilemma, a well known and standard problem in game theory which involves two players who must decide without prior knowledge of the other player's decision. A different problem in which neither player is actually making a decision, but instead is controlled by a deterministic algorithm, and in which both players, by looking at source, are able to know the other's decision in advance, is certainly an interesting puzzle to consider, but it has next to nothing in common with the Prisoner's Dilemma besides a payoff matrix.
1JoshuaZ
Prisoner's paradox is another term for the prisoner's dilemma. See for example this Wikipedia redirect. You may want to reread what I wrote in that light. (Although there's some weird bit of illusion of transparency going on here in that part of me has a lot of trouble understanding how someone wouldn't be able to tell from context that they were the same thing.) No. The problem of what to do is actually closely related when one has systems which are able to understand each others source code. It is in fact related to the problem of iterating the problem. In general, given no information, the problem still has relevant decision theoretic considerations.
3Perplexed
I'm curious why you assert this. Game theorists have a half dozen or so standard simple one-shot two person games which they use to illustrate principles. PD is one, matching pennies is another, Battle of the Sexes, Chicken, ... the list is not that long. They also have a handful of standard ways of taking a simple one-shot game and turning it into something else - iteration is one possibility, but you can also add signaling, bargaining with commitment, bargaining without commitment but with a correlated shared signal, evolution of strategies to an ESS, etc. I suppose that sharing source code can be considered yet another of these basic game transformations. Now we have the assertion that for one (PD is the only one?) of these games, one (iteration is the only one?) of these transformations is closely related to this new code-sharing transformation. Why is this assertion made? Is there some kind of mathematical structure to this claimed relationship? Some kind of proof? Surely there is more evidence for this claimed relationship than just pointing out that both transformations yield the same prescription - "cooperate" - when there are only two possible prescriptions to choose among. Is the code-sharing version of Chicken also closely related to the iterated version? How about Battle of the Sexes?
0JoshuaZ
So I'm going to need to repeat my earlier disclaimer that I'm far from my area of expertise. But the basic idea is that iterating games gives you a probabilistic estimate for what the underlying code looks like (assuming some sort of nice distribution on potential source code such that in general simpler code is more likely than complicated code). Unfortunately, I don't know any details of this approach beyond its existence but it should apply to other games like Chicken also.

We are at the very beginning of time for the human race. It is not unreasonable that we grapple with problems. But there are tens of thousands of years in the future. Our responsibility is to do what we can, learn what we can, improve the solutions, and pass them on.

Richard Feynman

Farnsworth A: You people and your slight differences disgust me. I'm going home. Where's that blue box with our universe in it?

Farnsworth 1: Oh, you'd like to get back to your evil universe, wouldn't you? And destroy your box with our universe inside it.

Farnsworth A: Nonsense! I would never do such a thing unless you were already having been going to do that!

--Futurama

The important thing is not to shout at this point, Vimes told himself. Do not ... what do they call it ... go spare? Treat this as a learning exercise. Find out why the world is not as you thought it was. Assemble the facts, digest the information, consider the implications. Then go spare. But with precision.

  • Terry Pratchett, Thud!

[I have had cause to apply this one recently. It particularly resonated to see it in the book just now.]

1Alicorn
This seems to be missing, at minimum, some punctuation. Edit: Moot.
1David_Gerard
Ellipses eaten by cut'n'paste. Fixed. Thank you :-)
0gwern
For the benighted Yanks among us: * http://en.wiktionary.org/wiki/go_spare * http://www.urbandictionary.com/define.php?term=Go%20Spare (The secondary Wiktionary definition doesn't seem to convey the depth of anger that Urban Dictionary and various citations I looked at did.)

There are more fools than knaves in the world, else the knaves would not have enough to live upon.

-Samuel Butler

Beware of no man more than of yourself; we carry our worst enemies within us.

Charles H. Spurgeon

Not always. I know someone who narrowly avoided Auschwitz who would beg to differ; her worst enemies were definitely external.

Your comment raises a very delicate point and I'm not sure that I am tactful enough to make it clearly.

Zooming out to get a broader view so that we can notice what usually happens, rather than the memorable special case, we notice that most Germans were enthusiastic about Hitler, all the way from 1933 to 1941. It is hard to reconstruct the reasons why. Looking at the broad picture we get a clear sense of people being their own worst enemies, enthusiastically embracing a mad leader who will lead them to destruction.

The message that history is sending to Alan is: if you had been a young man in Germany in 1933 you would have idolized Hitler. There are two ways to respond to this sobering message. One is to picture myself as an innocent victim. There were plenty of innocent victims, so this is easily done, but it dodges the hard question. The other response is to embrace the LessWrong vision and to search for ways to avoid the disasters to which self-deception sentences Man.

You're right, and I think that the reason it's so hard to make that point tactfully is because of how scary it is. If we go down that line of thought honestly, we can imagine ourselves firing up the ovens, or dragging manacled people into the belly of a slave ship, and feeling good about it. This is not a comfortable idea.

But there's another, more hopeful side to this. As MartinB points out, it's possible to understand how such monstrous acts feel to the people committing them, and train yourself to avoid making the same mistakes. This is a problem we can actually attack, as long as we can accept that our own thoughts are fallible.

(On a lighter note: how many people here regularly catch themselves using fallacious logic, and quickly correct their own thoughts? I would hope that the answer is "everyone", or at least "almost everyone". If you do this, then it shows that you're already being significantly less wrong, and it should give a fair amount of protection against crazy murderous ideologies.)

3MartinB
I doubt that it is. You find similar idolizations of leaders in many places. The general principles can be understood, and I think are by now. For the special case of nazi-germany you have the added bonus of good documentation and easy availability of contemporary sources.
-1roland
I'm a big fan of lesswrong yet I think it falls short because it lacks any concrete steps taken in the direction of being more rational. Just reading interesting posts won't make you a rationalist.
6simplicio
It's true that just reading posts won't make you more rational very fast. But thankfully, that is not the extent of LW - it is also encouraging people to respond to arguments they see, in a social context that rewards improving skills very highly. We are sort of practicing "virtue rationality" here, if you will. Once you have truly assimilated the core ideas of LW, to the point where they're almost starting to feel like cliches, you simply cannot HELP but to apply them in everyday life. For example, "notice when you're confused" saved my bacon recently: I was working on a group engineering project (in university) which was more or less done, but there was some niggling detail of interfacing that didn't sit well with me. I didn't know it was wrong; I just had a weird sensation of butterflies and fog every time I thought about that aspect. In the past I have responded to such situations with a shrug. This time, inspired by the above maxim, I decided to really investigate, at which point it became clear that our design had skipped a peripheral but essential component. I can cite more personal examples if you like. The trouble with noticing such instances is that once a skill is truly digested, it doesn't have a little label that says "that skill came from LessWrong." It just feels like the obviously right thing to do.
[-]anonym140

Can't you say "not always" about pretty much any quote? They aren't meant to be taken as universal truths that apply to all people and all circumstances across all of time ;-).

0[anonymous]
There's nothing worse than nitpicking a hyperbole.
3MichaelVassar
True, but barely. For how long do you think she would have had to plan and execute fully rationally in order to prevent Auschwitz. I think that it would have been a lot of work, but not insanely much work if done honestly.
2NancyLebovitz
? Do you mean avoiding getting sent to Auschwitz, or preventing the Holocaust? Escaping was something of a gamble. It probably wasn't obvious that fleeing to France wasn't good enough.
4MartinB
I guess when the guys that hate your guts get into power, is a good time to start packing. But after a decent time in the 20s, and lots of history, and many people of jewish decent being educated, and involved in the society, it is hard to see the signs. Jews have served in the 1. world war, and rightfully, and completely saw them self as Germans. Getting banned from professions came later, limits to who can marry whom and so on. It reminds me of the story of the frog that slowly gets heated up in water. Each step seems only a little worse than the other, so one thinks it might fade away. One should also keep in mind that racism and sexism was more widely spread in these days. Jews were not particularly welcomed in the US or elsewhere. The horror of Auschwitz was never announced. on each step there was talk of relocation. That includes the officials. No one imagined that a cultured people would be so barbaric. For a fictional presentation on how to turn up the heat the original V miniseries is pretty good.
6Pavitra
I'm a homosexual atheist living in the United States, and apparently people take the teabaggers seriously enough to vote for them. Should I move? Considered under the categorical imperative, this strategy seems like it would lead to people clustering themselves into super-fanatical cliques, which strikes me as undesirable. In particular, it would become harder and harder for anyone to change their mind, and thus harder for human knowledge to progress. Note also that, if the liberal Americans are the first to leave, the trigger-happy neocons get to keep control of the heavily nuclear-armed country.
2MartinB
I will tell you in hindsight. The move or change decision is an interesting one. For German Jews it was obviously better to leave. I would guess that many dissidents in islamic countries are also better off being alive in exile. Edit: Formatting
5NancyLebovitz
As I understand it, a good many German Jews had the amount of warning and the resources to get out. Polish Jews were caught more by surprise and (I think) were generally less well off, and most of the Holocaust happened there. Perhaps we should have a discussion about making high-stakes urgent decisions under conditions of great uncertainty.
1MartinB
I happen to be German, currently live in Nuremberg, and finally got around to visit Auschwitz last year. But i do not know the relation of people that flew and people that stayed. Fleeing also involed the ability to pay for the ticket. I probably read some about that, but forgot. It is true that the killings mostly happened in the east. But quite many were deported there just for this purpose.
1NancyLebovitz
Wikipedia: Over 90% of Polish Jews were killed, and about 75% of German Jews. Until I checked, I didn't realize that the proportion of German Jews who were killed was that high. I didn't have a specific number in mind, I think I was just giving more attention to the idea of those who'd escaped.
0MartinB
Oh, and the difference between people that flew at a suitable time vs. the number of people who survived. The later includes people who were deported but not killed, so the former is even lower. If you haven't read it yet. I found the comic from Art Spiegelman called 'MAUS!' pretty intense and interesting.
0MartinB
Yes. Not having been there limits imagination. Pre WW2 jews were as common as they are now in the US (or maybe more.) Now you will not find that many. All people of Jewish decent i know are not from Germany. In the last year I stumbled over genocides. This being the most unexpected evil I found..
1NancyLebovitz
This is the one that surprised me.
0MartinB
Why?
0NancyLebovitz
Failure to think about the British record in Ireland, perhaps. Not thinking about the mere size of India, so that if things go wrong, huge numbers of people die. So why did the Canadian atrocities against First Peoples surprise you?
1MartinB
I am not so much surprised with the atrocities against natives. Those are common. It is a bit surprising to learn it about Canada, because the country has a good reputation and its history is not that well known. But: Those were schools! Schools with a death rate is so much against anything that I consider a school to be about. Its just wrong. It just looks like a relabeled death camp, and that defeats the point of education. That amount of ignorance is just mindblowing.
0MartinB
Depends on your level of paranoia. They might still be after you. Lets have that discussion.
2NancyLebovitz
It's not just them being after you-- sometimes medical decisions fall into the same category.
1MartinB
Maybe we should make a collection of realistic situations. From a little thought I guess one tries to minimize damage, or maximize expected welfare. Both strategies need some calibration for realism.
0NancyLebovitz
Yes, on the calibration-- in particular, how do you maintain focus to do the best you can with the information you've got? What sorts of information do you need? I'm haunted by a quote from a holocaust survivor who said that he would have done things differently (presumably fled early) if he'd "had the soul of a poet". Hindsight is 20/20, but predicting from group emotional trends is sometimes part of what's necessary.
3MichaelVassar
Prevent the Holocaust.
2NancyLebovitz
How do you think that could have been done?
1MichaelVassar
General principles. Doing things isn't ever that difficult relative to the psychological capabilities we casually assume ourselves to possess. We then fail to update correctly and include that goals are difficult rather than concluding that over long time horizons we don't work the way we very casually seem to over periods of a few minutes.
0NancyLebovitz
On the other hand, the universe doesn't guarantee that apocalypse is scaled to your abilities. It's plausible that the Holocaust could have been averted if people had done more to optimize their efforts against it, but by no means guaranteed.
5simplyeric
"And when they came for me, there was no one left to speak out for me." —Martin Niemoeller I think that quote speaks a little about the worst enemies within us, in purely clinical terms, that what's in the best interest of those with whom you don't necessarily explicitly associate yourself may also be in your own best interest. The thing to keep in mind about the Jewish Holocaust is that it wasn't particularly unusual. It was unusual mostly in its location: it was rare to carry out such large scale atrocities ''in Europe''. Exterminations had been carried out by various states upon people in every other part of the world. Some were absolute, and entire races were exterminated. Hitler had great admiration for how the United States dealt with its native population. Sweden exterminated slaughtered whole groups in Africa. The list is not as short as we'd like it to be. An interesting (and depressing) book: Exterminate All the Brutes by Sven Lindqvist What I took from this book is that the enemy that is the holocaust situation is within us. The Jewish Holocaust was (unfortunately) not an outlier, but rather was/is in our culture or genes or humanity (I'm not sure I know which, although I tend towards the genetics).
1NancyLebovitz
What is unusual (I think) about the Jewish Holocaust is that it wasn't part of a conquest. Jews were very well integrated into German society, and had never been at war with it. Any other similar cases?
0simplyeric
Maybe a more salient example than my integrated Native Americans: Protestants v. Catholics. In certain circumstances it was simply war and/or strife. ("simply") But, in situations where both groups were fully native, there were situation where one group would try to eliminate the other through legislation, deportation, and also extermination.
0Jayson_Virissimo
Ukrainians and Poles in the U.S.S.R.? I guess it would depend on your definition of "conquest".
0NancyLebovitz
And possibly also "integrated"-- my impression is that Jews in Germany were less geographically concentrated, but even if true, that might be reaching for an argument.
0simplyeric
I think that within the subset of United States's aggression against the Native American population, there were many instances where fully integrated people were subsequently persecuted and eliminated. Some of it was at the "frontiers", yes. But some of it was shopowners, millers, brewers...people who had fully adapted and in fact thrived within the europeanized colonies and later states. This was still happening in the 1950's and 60's as well, with the flooding of native lands in the Dakotas, etc, where fully "Americanized" communities were eliminated through forced relocation.
-5[anonymous]
[-]ata70

Rationality quotes: very many from @BadDalaiLama on Twitter.

(Edit: there's also this handy archive.)

5cupholder
This one felt quite LW-relevant: It's good to be reminded now and then that dollars are not, in fact, utilons.
1shokwave
The natural logarithm of dollars is a pretty good approximation of utilons, assuming you like candy-bars.
6NihilCredo
With some constraints, of course.
2Unnamed
Here's some evidence from Stevenson & Wolfers that happiness/life satisfaction is proportional to the log of income: blog post, pdf article.
2Psy-Kosh
How does ln(dollars) approximate utilions? It's obvious that utilions are generally not fully linear in dollars, and they're certainly not equivalent, but how does the log of dollars, specifically, approximate utility?
1shokwave
If there is some mathematical reason why, I would love to know. I was going off the observation that the natural logarithm approximates the kind of diminishing returns that economists generally agree applies to the utility of wealth. This means that, very roughly, the logarithm of dollars is the 'revealed preference' utility. It was actually more of a joke about that assumption, because it suggests that a 50 dollar meal is preferred four times as much to a 3 dollar candy bar - a bit odd, but perfectly natural if you like candy bars.
3JoshuaZ
Well, log does that. But so does square root also. Lots of functions have diminishing marginal returns.
2b1shop
I can think of two good reasons to model diminishing returns with the natural log. Logs produce nice units in the regression coefficients. A log-lin function (that is -- log'd dependent, linear independent) says that a percent increase in X results in a unit increase in Y. Similar statements are true for lin-log and log-log, the latter of which produces elasticities. y=ln(x) and y=sqrt(x) will both fit data in a similar manner, so it makes sense to go with the one that makes for easy interpretation. Additionally, the natural log frequently shows up in financial economics, most prominently in continuous interest but also notably in returns, which seem to follow the log-normal distribution.
3Manfred
Of course, there's the problem with pathological behavior near 0. Or the utility of money could quite reasonably be bounded.
0shokwave
Hmm. If we grab some study data on wealth's mathematical relationship with utility, we might be able to decide what function best approximates it. As it is, yeah, there is no reason to prefer log to square root to anything other function.
0Psy-Kosh
Oooh, okay. Diminishing returns, certainly. Just not obvious that it would be "log" or near that. :)
2free_rip
Nice link. My favorite: In a democracy, the poor have the same power as the rich, but the rich can buy advertising, which the poor are suckers for.

Isaac Newton's argument for intelligent design:

Were all the planets as swift as Mercury or as slow as Saturn or his satellites; or were the several velocities otherwise much greater or less than they are (as they might have been had they arose from any other cause than their gravities); or had the distances from the centers about which they move been greater or less than they are (as they might have been had they arose from any other cause than their gravities); or had the quantity of matter in the sun or in Saturn, Jupiter, and the earth (and by consequ

... (read more)
7Tyrrell_McAllister
Here's another Newton ID quote. This one complements PeterS's because the true naturalistic explanation requires physics that was not implicit in Newton's mechanics. —Isaac Newton, Four Letters From Sir Isaac Newton To Doctor Bentley Containing Some Arguments In Proof Of A Deity.
0wedrifid
Elements of this argument make an error related to numberplates. I'm surprised this was received so (+4) positively.
5Perplexed
I voted up both Newton quotes because they show how a very smart man can make a very plausible argument which is nevertheless very wrong. And the reason Newton failed to guess the rather simple explanation is that he observed a solar system that was stable and unchanging and assumed that it must always have been stable and unchanging since the creation. His "biases" just didn't allow him to imagine an evolutionary model of planet formation by accretion from a more-or-less random initial state. Nowadays of course, we tend to invent evolutionary or historical explanations for everything. We don't even limit ourselves to explaining the origins. We go on to predict how things will likely come to a contingent historical end ... or should I refer to it as our next great adventure?
2[anonymous]
Second to this. The planets that remain in sequence and orbit survived the transition from entropy to stability in a way that didn't result in them being ejected or destroyed. Their presence represents them making it through the pachinko machine of amalgamated physical parameters, not intentional design. Newton's inferences were like assuming a gold tooth has mystical properties because I've put you through a woodchipper and it's the only thing that came out the other end. There is so much to understand about the internals of the machine before you make any solid judgments about the inputs and outputs.
0magfrump
I thought it was obviously ironic, since planets do actually move in ellipses and general conic sections; Newton makes a falsifiable claim in favor of ID and it is clearly false.
2Sniffnoy
Wait, something seems wrong here. Newton knew the planets moved in ellipses. Probable conclusion: He was just referring to the low eccentricity of these ellipses?
0Tyrrell_McAllister
I think that the issue was the number of planets. If you had just one planet orbiting the sun, that orbit would be a nice stable one. But if you have multiple bodies orbiting the sun, their paths will interfere chaotically. I think that Newton expected that, in general, you would get wildly erratic orbits, with some planets being thrown clear of the system altogether. As I understand it, he expected such catastrophes to be inevitable, unless you started with a very carefully-selected initial state. God was then necessary to explain how the solar system started out in such an improbable state. But in fact Newton just lacked the mathematical sophistication to see that, according to his own theory, typical initial arrangements could result in systems that are stable for billions of years.
0PeterS
Numberplates?
1wedrifid
"The chance that the numberplate of my first car was EIT411 is one in a whole lot. Wow! It happened! There must be a God!" (crudely speaking.) This seems to be relevant to, for example, yabbering on about the exact speeds of Saturn et. al. The Saturns that were going the wrong speed all fell in to the sun (or cleared off into space.)
1PeterS
Oh... I in no way endorse the above argument! Pierre-Simon Laplace's, a century or so after Newton, gave a naturalistic model of how the Solar System could have developed. "Rationality quotes" is not only about sharing words of wisdom, but also words of folly.
0wedrifid
:) I certainly wasn't intending to accuse you.

If oxen and horses and lions had hands and were able to draw with their hands and do the same things as men, horses would draw the shapes of gods to look like horses and oxen to look like oxen, and each would make the gods’ bodies have the same shape as they themselves had.

Xenophanes

8JoshuaZ
I'm not sure this makes sense. Empirically many human cultures have deities that are shaped like animals.
0fortyeridania
Voted up. My quibble is that gods are often anthropomorphic in mind, if not in body.
6shokwave
More likely they would write a treatise on how God wants them to keep pulling carts around.
5Larks
Well, the Egyptians had animal-headed gods.
0Tiiba
With human bodies.
4simplyeric
There might be a strong chance that horses and other animals would draw their gods as having human form. Humans tend to protray their gods as being either equal or higher than humanity. Animist gods are protrayed as having characteristics that surpass humans: speed, wisdom, patience, etc. based on the characteristics of that animal. Alternately, sun gods, storm gods, etc.: higher powers. Some wild horses would have horse gods or weather gods or wolf gods. Some might have human gods, depending on their interaction with humanity. I'd imagine that domesticated horses would have human gods, some benevolent and some malignant, or both. And some domesticated horses would go "through the looking glass" and develop a horse-god of redemption, with prophecies of freeing them from the toil and slavery of domestication, based on some original downfall of horse-dom that led to them being subservient to humans. Or something like that.
0nazgulnarsil
this should at the very least be turned into a short story.
2majus
Charles de Secondat, Baron de Montesquieu "If triangles had a god, he would have three sides." [Lettres Persanes, no 59]
6Larks
Surely he would be circular?
[-]Rain70

I am still learning.

-- Michaelangelo's motto

From Brandon Sanderson's Mistborn series:

"Certainly, my situation is unique," Sazed said. "I would say that I arrived at it because of belief."

"Belief?" Vin asked.

"Yes," Sazed said. "Tell me, Mistress, what is it that you believe?"

Vin frowned. "What kind of question is that?"

"The most important kind, I think."

Vin thought, then shrugged. "I don't know what I believe."

"People often say that, but I find it is rarely true."

Overall, however, we've done better by avoiding dragons than by slaying them. -Warren E. Buffett

The course of human progress staggers like a drunk; its steps are quick and heavy but its mind is slow and blunt

-Jesse Michaels of Operation Ivy

Posted because it's a useful and evocative metaphor: the drunk feels himself leaning or falling in one direction, and puts his foot down in that direction to steady himself. If he doesn't step far enough, he is still leaning in the same direction, and he steps again. In this way we can make fantastic progress in directions we don't like while getting further away from the ways we did want to go.

I just came across this and thought it was a pretty funny dialogue: "Reality is that which does not go away upon reprogramming." (Check the first 4 comments here: Chatbot Debates Climate Change Deniers on Twitter so You Don’t Have to)

This is of course a paraphrase borrowed from Philip K. Dick's famous statement:

Reality is that which, when you stop believing in it, doesn't go away.

I shared this on another website and got this comment:

Heh, that's one way to pass the Turing Test. Don't make your bot smarter, make it seek out dumb people.

3Tuna-Fish
This has been done for a while. A few years ago there was some noise about a russian chatbot which impersonated a good-looking girl and tried to scam people to give personal information and/or money. Every time it succeeded, it passed the turing test.

'Tis with our Judgments as our Watches, none
Go just alike, yet each believes his own.

Pope, Essay on Criticism

Research must contrive to do business at a profit, by which I mean it must produce more effective scientific inquiry than it expends. No doubt it already does so. But it would do well to become conscious of it's economic position and contrive ways of living upon it. -CS Peirce

And should you ask yourselves, "How can we know that the oracle was not spoken by the Lord?" -- if the prophet speaks in the name of the Lord and the oracle does not come true, that oracle was not spoken by the Lord; the prophet has uttered it presumptuously: do not stand in dread of him.

Deuteronomy 18:20-22

7PhilGoetz

half of being smart is knowing what you're dumb at

solomon short (david gerrold's fictional character)

Out-of-sample error equals in-sample error plus a penalty for model complexity

Y.S. Abu-Mostafa, in explaining the VC inequality of PAC learning.

0[anonymous]
This quote leaves out the 'P' part of "PAC"! It should really read,

There are times I almost think
Nobody sure of what he absolutely know
Everybody find confusion
In conclusion he concluded long ago
And it puzzle me to learn
That tho' a man may be in doubt of what he know,
Very quickly he will fight...
He'll fight to prove that what he does not know is so!

~ A Puzzlement, The King and I

Scooping the Loop Snooper

an elementary proof of the undecidability of the halting problem

by Geoffrey Pullum

No program can say what another will do. Now, I won't just assert that, I'll prove it to you: I will prove that although you might work till you drop, you can't predict whether a program will stop.

Imagine we have a procedure called P that will snoop in the source code of programs to see there aren't infinite loops that round and around; and P prints the word "Fine!" if no looping is found.

You feed in your code, and the input it needs, and

... (read more)

The Universe behaves according to its own laws.

Talmud, Avoda Zara 54b

Most of the founding Zetas members–the original 40–were trained elite soldiers who received instruction in radio communications, counter-insurgency and drug-interdiction. But the Army forgot to add a few ethics lessons into the education mix. And that was a big fucking mistake.

The Mexican Drug War in One Lesson: Know Your Zetas!

"...for we judge ourselves by what we feel capable of doing, while others judge us by what we have already done."

--Henry Wadsworth Longfellow, Kavanagh ch. 1

0Document
Related: correspondence bias.

I chose and my world was shaken, so what? The choice may have been mistaken; the choosing was not.

Sunday in the Park with George, by Stephen Sondheim

I can't keep rationalizing away everything like this, or I'm gonna die!

-- Gordon Freeman, kind of

4cousin_it
Another "rationality quote" from the same video:
[-]ata30

A universe that needed someone to observe it in order to collapse it into existence would be a pretty sorry universe indeed.

— Randall Munroe, xkcd – Mutual

1Spurlock
That comic gets bonus points for nice use of Hofstadter-ian strange loop.

Q: Don't you think sire, that the TV commercials should look more like real life?

A: On the contrary! I think life should be more like TV commercials.

  • Benny Hill
[-][anonymous]30

del

5jimmy
That doesn't sound right. To me, it seems like: (Philosophy -> Science) and (Art -> Engineering).

Don't feed the trolls, people, this is not reddit.

Beware of finding a Jesus entirely congenial to you.

-- The Jesus Seminar

(Developed in the context of biblical interpretation, of course. But despite my nontheism, I've found the principle behind it to be widely applicable.)

For the young who want to by Marge Piercy

The poem is mostly about not being recognized as having a magical ability to do things until after you've succeeded. I'm just posting the link because it's more trouble than it's worth to make the line breaks show up properly.

"Mortal danger is an effective antidote for fixed ideas."

Erwin Rommel, The Rommel Papers (1982) edited by Basil Henry Liddell Hart http://en.wikiquote.org/wiki/Erwin_Rommel#Sourced

0jaimeastorga2000
This reminds me of the phrase "nobody learns faster than someone who is being shot at". Considering all the technological research done in war time, there seems to be a good point about motivation.

"But building your life's explanations around science isn't a profession. It is, at its core, an emotional contract, an agreement to only derive comfort from rationality."

-Robert Sapolsky, in a essay reply to "Does science make belief in God obsolete?"

[-][anonymous]20

"All of us, grave or light, get our thoughts entangled in metaphors, and act fatally on the strength of them."

---George Eliot, "Middlemarch"

0Craig_Heldreth
Somebody else read the comments section in Sapolsky's New York Times op ed today. His column had a rough explanation of human oddities explained as evolutionary adaptations. link (If you sort the comments by largest approval rating there are several interesting ones.)

When one person suffers from a delusion it is called insanity; when many people >suffer from a delusion it is called religion.

~ Robert M. Pirsig

Now the actual quote's out of the way, here's my version: when one person suffers from a delusion it is called insanity; when many people suffer from a delusion it is called society.

1MartinB
Or friendship, or marriage, or all kinds of other things.

Truth will sooner come out from error than from confusion.

Francis Bacon

3Richard_Kennaway
Duplicate, twice.
0[anonymous]
Wait, is there a convenient way to check and see which ones have been posted so far? You know, in case you don't have an encyclopedic memory of all the quotes posted over the last 18 months?
1Richard_Kennaway
Use the search box to check any quotation you're thinking of posting.
0anonym
Oops, I'm sorry. I must have forgotten to check that one first.
0simplyeric
I'm of the mind that politically, in the US at least, we don't seem to learn from this. The truth is, indeed, revealed....but the confusion remains and the errors continue. There are many who disagree with me about that... but that's because they're confused AND in error. (ok ok I kid on that last part...)
[-]Jack10

Awkward.

[-][anonymous]10

Not a quote about rationalism, but probably relevant to Less Wrong:

Pure Death

We looked, we loved, and therewith instantly

Death became terrible to you and me.

By love we disenthralled our natural terror

From every comfortable philosopher

Or tall, grey doctor of divinity:

Death stood at last in his true rank and order.

It happened soon, so wild of heart were we,

Exchange of gifts grew to a malady:

Their worth rose always higher on each side

Till there seemed nothing but ungivable pride

That yet remained ungiven, and this degree

Called a conclusion not to be denied.

T

... (read more)
2gwern
I read this one last week: "When human beings found out about death They sent the dog to Chukwu with a message: They wanted to be let back to the house of life. They didn't want to end up lost forever Like burnt wood disappearing into smoke And ashes that get blown away to nothing. Instead, they saw their souls in a flock at twilight Cawing and headed back for the same old roosts (The dog was meant to tell all this to Chukwu)..." --"A Dog Was Crying Tonight in Wicklow Also", Seamus Heaney, pg 66, The Spirit Level (1996)
[-]SK200

All my confidence comes from knowing God's laws.

-- Talib Kweli (substitute "nature" for "God")

8Risto_Saarelma
I don't think it would be a good idea to take a Carl Sagan quote and add a 'substitute "God" for "nature"' postscript. I don't think this is a good idea either.
4SK2
Talib Kweli is nonreligious, so I'm not changing the meaning of the quotation. "God" is often used poetically. Example: Albert Einstein Even if Kweli were religious the point would not be to put words in his mouth, but to reapply a beautiful quotation to another context where it is meaningful.
0smdaniel2
reapplying it to another context changes the meaning. because of einstein's explicitly stated opinions on the meaning of God (and the Lord), we can understand his meaning to be synonymous with that of nature and its order. "I believe in Spinoza's God who reveals himself in the orderly harmony of what exists, not in a God who concerns himself with the fates and actions of human beings." "I do not believe in a personal God and I have never denied this but have expressed it clearly. If something is in me which can be called religious then it is the unbounded admiration for the structure of the world so far as our science can reveal it. " - 1936 Talib Kweli, on the other hand, hasn't given us a clear opinion of his thoughts on the term God. There is no evidence for us to assume that the meaning he gives to the term God would fit in the context of this quote.
[-][anonymous]00

Intellectual brilliance is no guarantee against being dead wrong.

David Fasold

[-]neq100

Justice is an artefact of custom. Where customs are unsettled its dictates soon become dated. Ideas of justice are as timeless as fashions in hats.

-John Gray, Straw Dogs

4jfm
~ Epicurus, Principal Doctrines
4Jayson_Virissimo
-Anthony de Jasay, Inspecting the Foundations of Liberalism Conventions against torts like murder and theft are older than civilization. I think it is a safe bet they will still be around in a thousand years.
[-]neq100

Who has not experienced the chilling memory of the better things? How it creeps over the spirit of one's current dreams! Like the specter at the banquet it stands, its substanceless eyes viewing with a sad philosophy the make-shift feast.

-Theodore Dreiser, The Titan

[-][anonymous]00

There are times I almost think, Nobody sure of what he absolutely know. Everybody find confusion, In conclusion he concluded long ago. And it puzzle me to learn , That tho' a man may be in doubt of what he know, Very quickly he will fight... He'll fight to prove that what he does not know is so!

[-][anonymous]00

There are times I almost think Nobody sure of what he absolutely know Everybody find confusion In conclusion he concluded long ago And it puzzle me to learn That tho' a man may be in doubt of what he know, Very quickly he will fight... He'll fight to prove that what he does not know is so!

~ A Puzzlement, The King and I

"However insistently the blind may deny the existence of the sun, they cannot annihilate it. " - D. T. Suzuki

6wedrifid
Want to bet? (At stakes of a few thousand galaxies worth of energy and negentropy. It's not going to be cheap to win this bet! I'm not too comfortable with the whole making myself blind thing either but I guess I can rectify that once I finish deploying the antimatter disruptor beam.)
7Paul Crowley
"Since the beginning of time, man has yearned to destroy the Sun."
0[anonymous]
"Since the beginning of semester, man has yearned to destroy the Sun."
4wedrifid
I just noticed that I implicitly assumed that it would have to be me that blinded himself. What sort of nefarious sun destroying intergalactic mastermind would I be if did foist that role upon a henchman?
1Snowyowl
You're going to have trouble destroying the Sun if you don't believe it exists.
8Larks
He only has to deny that it exists. Alternatively, he could lock himself onto a sun-destroying path, and then forcibly do an unBayesian update away from the existence of the sun. Alternately, he could interpret the sentence literally, note that 'not at all' is a level of insistence, deny the existence of the sun not at all, and then destroy it.
1yoj1mbo
You are mistaking the map for the territory. It doesn't matter if its a quark-pair or a hyper colossal cosmic structure gravitationally influencing everything in the universe; if a condition is present, then it has effect.
0wedrifid
That settles it. I'm going to recruit Chewbacca as my Mook Luitenant. There can be no other choice.
1free_rip
The replies are better quotes than the original.
0MartinB
Humans are blind to all kinds of things. Radiation for one. But it still can be detected and controlled. A civilization of blind people would eventually build detecting equipment and learn to control the real world.
-2Document
As the saying goes: "In a world of blind people, c would be the speed of heat.". But I don't think anyone believes that anyone actually denies the sun's existence.
[-]gwern-40

"Normal humans don't interest me. If anyone here is an alien, a time traveler, slider, or an esper, then come find me! That is all."

--The Melancholy of Haruhi Suzumiya, vol 1

1NihilCredo
Rationality?
[+][anonymous]-60