Another month, another rationality quotes thread. The rules are:

  • Please post all quotes separately, so that they can be upvoted or downvoted separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)
  • Do not quote yourself.
  • Do not quote from Less Wrong itself, HPMoR, Eliezer Yudkowsky, or Robin Hanson. If you'd like to revive an old quote from one of those sources, please do so here.
  • No more than 5 quotes per person per monthly thread, please.
  • Provide sufficient information (URL, title, date, page number, etc.) to enable a reader to find the place where you read the quote, or its original source if available. Do not quote with only a name.
Rationality Quotes July 2014
New Comment
198 comments, sorted by Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings
[-]satt600

Charles II is said to have himself toyed with the philosophers, asking them to explain why a fish weighs more after it has died. Upon receiving various ingenious answers, he pointed out that in fact a dead fish does not weigh anything more.

— Robert Pasnau, "Why Not Just Weigh the Fish?"

6gwern
I think Alistair might have mangled the story there. There does seem to be a Charles II/fish/weight story, but about a completely different weight - in water, not postmortem: https://gwern.net/doc/philosophy/epistemology/1948-oesper.pdf Fortunately, while the question Charles II posed in this version is considerably clunkier, the upshot remains the same, so there are much worse leprechauns... (Although the sourcing here is still thinner than I'd like and may not be the original: no date is given, but Schönbein was born in 1799 and Charles II died in 1685, and an 1842 publication still leaves at least 157 years between the latest the story could've happened and this exact publication. But I'll leave it to someone else to try to track it further back.)
6Gurkenglas
Well that's not fair, generating hypotheses that fit an observation is sometimes useful, as Harry&Draco did when finding some that fit magic getting weaker. The philosophers, when acting as oracles, should not have to entertain the possibility that the assumptions given to them are wrong.

In general, considering how many wrong assumptions there are, I think it's good practice to check the assumptions one has been handed.

On the other hand, it's probably not fair for kings to play that sort of game.

1satt
Amplifying this: I'm fairly sure that (paraphrasing) "philosophers are useful for exposing other people's wrong assumptions" is one of the stock justifications for philosophizing!
[-]Pablo500

Prediction, not narration, is the real test of our understanding of the world.

Nassim Nicholas Taleb, The Black Swan: The Impact of the Highly Improbable, New York, 2007, p. 133

We have to reinvent the wheel every once in a while, not because we need a lot of wheels; but because we need a lot of inventors.

-- Bruce Joyce, as quoted by Michael Serra in Discovering Geometry

[-]Pablo420

Another possibility is that our intuitive sense of justice is a set of heuristics: moral machinery that’s very useful but far from infallible. We have a taste for punishment. This taste, like all tastes, is subtle and complicated, shaped by a complex mix of genetic, cultural, and idiosyncratic factors. But our taste for punishment is still a taste, implemented by automatic settings and thus limited by its inflexibility. All tastes can be fooled. We fool our taste buds with artificial sweeteners. We fool our sexual appetites with birth control and pornography, both of which supply sexual gratification while doing nothing to spread our genes. Sometimes, however, our tastes make fools of us. Our tastes for fat and sugar make us obese in a world of abundance. Drugs of abuse hijack our reward circuits and destroy people’s lives. To know whether we’re fooling our tastes or whether our tastes are fooling us, we have to step outside the limited perspective of our tastes: To what extent is this thing—diet soda, porn, Nutella, heroin—really serving our bests interests? We should ask the same question about our taste for punishment.

Joshua Greene, Moral Tribes: Emotion, Reason, and the Gap Between Us and Them, New York, 2013, p. 272

3[anonymous]
Hmmm... does he say something that goes beyond his condescending style? If he wants to criticize naive pleasure-hedonism, fine, but to talk of "best interests" without bothering to specify what he means by that phrase strikes me as an attempt to grab my utility function. At best it seems like a way to signal "Look how rational I am!" without having to admit to the banal motivation of wanting to sell books.

He's saying that our desire for punishment is potentially a lost purpose. How is that an attempt to grab the utility function?

Also, I'd be interested to hear why you read this as condescending; I don't see where you're coming from with that.

6[anonymous]
When, then, is it a lost purpose? Deterrence is a consequentialist mode of punishment. "I know what's good for you better than you do" is condescending until proven otherwise ;-). To know my good, you must do more than point out that I suffer from cognitive biases. If you say, "The sky is not green", you haven't demonstrated what I would call real knowledge, and you're not entitled to speak of others as ignorant for holding their own views (most of which will usually be, "the sky is blue", since it is blue). You haven't engaged with their views: you've merely stated a particularly obvious negative conclusion (eliminating part of the search space) as if it were a positive conclusion (identifying the portion of the search space where the truth actually lives). (This may be part of the same underlying complex of ideas that makes me prefer constructive mathematics.)

I'm not seeing it. To me, it seems like you're going to lengths to construe his writing in a way that you can take offense to. I don't actually think you are doing so, but your reframing is so distant from the tone I perceive that I can't understand what you are doing.

I read it as suggesting, in a fairly humble if flowery tone, that a number of other ancestral urges have been coopted for things that are demonstrably not in our best interest, and that desire for punishment is potentially on the same level. It's a suggestion worth investigating, in my view.

Or do you think that discouraging someone from drinking sugary sodas is on the same level? That could explain the disconnect.

Precise forecasts masquerade as accurate ones.

-- Nate Silver, The Signal and the Noise

How can you tell economists have a sense of humor? They use decimal points.

The distinction between precision and accuracy is one of the most useful distinctions I've learnt.

If your goal is to get at the truth, then accuracy is always the primary goal, and precision secondary. Indeed, it is quite dangerous to aim for precision first. This was also captured by Knuth, "premature optimization is the root of all evil."

Unfortunately, most people are convinced more easily by precision than by accuracy. Politicians and false prophets often employ this trick. Precision reflects confidence. Also, it is trivial to very whether a statement is precise; but incredibly difficult to verify if it is accurate.

A law professor who was a practicing defense attorney whom I talked with during my ordeal told me of an experiment he had done. He was at a dinner party and told people at one table that he was defending a man who was wrongly accused of molesting a child, and was met with shock and accusations of trying to free a monster. He told another table that he was defending a murder suspect whom he was convinced was guilty, and got, "Oh, that's sounds interesting. Tell me more."

Ray Atkinson on Quora

He told another table that he was defending a murder suspect whom he was convinced was guilty, and got, "Oh, that's sounds interesting. Tell me more."

My shock as an observer would have been the gross breach of confidentiality. Is that revelation grounds for a lawsuit, a criminal offense or merely grounds for disbarment? Regardless, it would have been a gross ethical violation on the same order of either of the other two offenses. Undermining the justice system like that is Evil (just an evil that is on the other end of the visceral disgust spectrum than the molestation.)

Is that revelation grounds for a lawsuit, a criminal offense or merely grounds for disbarment?

None of the above, really, unless you have so few murder cases that someone could plausibly guess which one you were referring to. I work with about 100 different plaintiffs right now, and my firm usually accepts any client with a halfway decent case who isn't an obvious liar. Under those conditions, it'd be alarming if I told you that 100 out of 100 were telling the truth -- someone's bound to be at least partly faking their injury. I don't think it undermines the justice system to admit as much in the abstract.

If you indiscreetly named a specific client who you thought was guilty, though, that could get you a lawsuit, a criminal offense, and disbarment.

6wedrifid
I will take you at your word that you could get away with making such disclosures. You are the lawyer and so the expert at judging what ethical violations people can technically get away with. I have to thank you for allowing me to update my expectations regarding the ethical standards I can expect from an average legal representative. I now know I will need to filter more aggressively myself and not rely on the system to provide what I would otherwise have taken to be the most rudimentary standards of integrity I need from someone in that role. (That's a sincere thankyou, not snide pettiness. I really was confused about what that social rules the legal subculture would at least enforce lip-service to adherence to.) 'Abstract' does not mean what you think it means. You are revealing concrete information that is slightly vague. You believe this is OK and as such can be trusted much less with private information. I still may (hypothetically) recommend someone use the services of someone with your beliefs about what constitutes acceptable disclosure of confidential information, but only if their fees are sufficiently low relative to their other competencies as to offset this liability. It wouldn't be alarming at all. It would sound exactly equivalent to "No comment". It'd sound like you were doing your job (albeit more awkwardly than if you had just shut your mouth and signaled tact). If you choose to speak about the guilt of your clients and choose to reveal anything less than the token "My clients are Resistance), not Spies" then you are disclosing personal information. Because mathematics. I usually abhor bullshit (advocacy with casual indifference to epistemic accuracy). Lawyers represent a notable exception, where unabashed advocacy for each side is the least bad option I know of for minimising injustice.

You're...welcome? For what it's worth, mainstream American legal ethics try to strike a balance between candor and advocacy. It's actually not OK for lawyers to provide unabashed advocacy; lawyers are expected to also pay some regard to epistemic accuracy. We're not just hired mercenaries; we're also officers of the court.

In a world that was full of Bayesian Conspiracies, where people routinely teased out obscure scraps of information in the service of high-stakes, well-concealed plots, I would share your horror at what you describe as "disclosing personal information." Mathematically, you're obviously correct that when I say anything about my client(s) that translates as anything other than a polite shrug, it has the potential to give my clients' enemies valuable information. As a practical matter, though, the people I meet at dinner parties don't know or care about my clients. They can't be bothered to hack into my firm's database, download my list of clients, hire an investigator to put together dossiers on each client, and then cross-reference the dossier with my remarks to revise their probability estimate that a particular client is faking his injury. Even if someo... (read more)

4wedrifid
Your ethical intent sounds fine but that is of limited use without competence. The sort of casual disclosure described in the ancestor anecdote would make me slightly downgrade my evaluation of the trustworthiness and social competence of any professional that works with sensitive information. Much like those observed casually gossiping about other people at inappropriate times will be silently downgraded as potential confidants. The overwhelming majority of minor ethical transgressions that we make will "do no harm". Some do. If the consequences were that easy to predict we wouldn't need ethical inhibitions in the first place.
4Mass_Driver
That's an important warning, and I'm glad you linked me to the post on ethical inhibitions. It's easy to be mistaken about when you're causing harm, and so allowing a buffer in honor of the precautionary principle makes sense. That's part of why I never mention the names of any of my clients in public and never post any information about any specific client on any public forums -- I expect that most of the time, doing so would cause no harm, but it's important to be careful. Still, I had the sense when I first read your comment six weeks ago that it's not a good ethical maxim to "never provide any information (even in the mathematical/Bayesian sense of "information") to anyone who doesn't have an immediate need to know it." I think I've finally put my finger on what was bothering me: in order to provide the best possible service to my clients, I need to make use of my social and emotional support structure. If I carried all of the burdens of my work solely on my own shoulders, letting all of my client's problems bounce around solely in my head, I'd go a little crazier than I already am, and I'd provide worse service. My clients would suffer from my peculiar errors of viewpoint. In theory, I can discuss my clients with my boss or with my assistants, but both of those relationships are too charged with competition to serve as an effective emotional safety valve -- I don't really want to rely on my boss for a dose of perspective; I'm too busy signalling to my boss that I'm competent. I think this is probably generally applicable -- I want my doctors to have a chance to chat about me (without using my real name) in the break room or with their poker buddies, so that they can be as stable and relaxed as possible about giving me the best possible treatment. Same thing with my accountant -- I'm much more concerned that my accountant is going to forget to apply for a legal tax exemption that'll net me thousands of dollars than I am that my accountant is going to leak det
3Mizue
I think there's a difference between "does no harm, because it had a substantial chance of doing harm, but someone got lucky", and "does no harm, and the chance of harm wasn't ever substantial to begin with".

Good noticing of confusion, I feel slightly ashamed of not picking up on that immediately.

8wedrifid
Thankyou, your paramedic IRC anecdote serves as one of the most practical triggers lesswrong has provided me.

The immediately preceding paragraph:

People wrongly accused of murder will have the charges dropped if the victim walks in the courtroom and testifies that it didn't happen. Accused child molesters get convicted even when the victim says that it didn't happen.

This is true. I hope the implied claim is "either people think differently about child molestation accusations than murder accusations OR necromancy is not possible".

[-]Shmi120

This link warrants a trigger warning:

multiple heartbreaking stories of false accusations ruining innocent people's lives

6Stabilizer
Why do I find these reactions highly counter-intuitive? That is, I would never have predicted that this is what people would say.
2wedrifid
I'm not sure. Is it could be that the details make the story sound apocryphal or contrived? Or is it that you find the underlying moral unbelievable? That is, do you expect that people's judgement of guilt is distorted heavily by the moral repugnance associated with the alleged crime?
8Stabilizer
Well, I find the attempt to save a falsely accused man to be much more morally admirable than the attempt to save a justly accused man. Indeed, the fact that child molestation is considered very morally repugnant and carries huge legal and social costs is part of the reason why I feel that any attempt to protect a man from false accusations of child molestation to be very admirable. To answer your question, I didn't expect (at least, not till now) people's judgement of guilt to be distorted so much by the moral repugnance of the alleged crime. If indeed people do distort this much, I should carefully rethink my understanding of moral intuitions.

This seems to be a very specific issue with child molestation in the United States, where there's a kind of weird none-dare-urge-restraint spiral around that topic for some reason.

5MarkusRamikin
I figure it's the "safe" reaction. Since child molestation is considered so much more repugnant, accidentally getting seen as having taken the side of a child molester (by allowing that he might be innocent when you don't really know enough to judge that) is a bigger social risk.
1A1987dM
I guess that was the whole point of the quote...
6MarkusRamikin
And here I could have sworn it also had something about the way people judge guilt and innocence.
2ike
Theory: People hear murder, and think, "Oh, the best you'll do is get them a couple years off, no judge will let a murderer go free." But when someone hears "molesting a child", they think, "They're probably guilty regardless of what you think, but you have a chance of convincing a jury and then unleash this monster on us." Of course, this is working on a subconscious level. I suppose the theory you're supposed to think of is "molestation is more graphic than murder"? That can be tested by substituting another, more graphic word for murder, e.g. "shot, exploded, cut up". (Trying not to get too graphic here.)

An expert is a man who has made all the mistakes which can be made in a very narrow field.

-- Niels Bohr, A Dictionary of Scientific Quotations

[-]gwern260

"Independence is for the very few, it is a privilege of the strong. Whoever attempts it enters a labyrinth, and multiplies a thousandfold the dangers of life. Not least of which is that no one can see how and where he loses his way, becomes lonely, and is torn piecemeal by some minotaur of conscience. If he fails, this happens so far from the comprehension of men that they cannot sympathise nor pity."

--29, Part 2: The Free Spirit, Friedrich Nietzsche's Beyond Good and Evil- Prelude to a Philosophy of the Future

8Stabilizer
While Nietzsche writes it beautifully, perhaps the simplified, layman version would be: "If you insist on refusing social obligations and violating social norms, then life becomes very hard: you will be lonely and your conscience will bother you a lot. If you fail---i.e. the pain of being outcast exceeds the benefits of independence---then no one will give a damn." (The last part is almost tautological; if you're lonely, then most people don't care about you. The exception might be when one writes one's experiences down, as Nietzsche probably did.)
8gwern
I was actually thinking it applied better to cranks than generic 'social obligations and norms'.
6D_Malik
Also in this vein is Sebastian Marshall's The Million Dollar Question.
2gwern
A very interesting post. I don't think that's more than a small fraction of what's going on, but I could believe it's a fraction.

If we all worked on the assumption that what is accepted as true is really true, there would be little hope of advance.

-- Orville Wright, http://wrightbrothers.info/quotes.php

[-]Pablo210

It takes […] what Berkeley calls a mind debauched by learning to carry the process of making the natural seem strange, so far as to ask for the why of any instinctive human act. To the metaphysician alone can such questions occur as: Why do we smile, when pleased, and not scowl? Why are we unable to talk to a crowd as we talk to a single friend? Why does a particular maiden turn our wits so upside-down? The common man can only say, “Of course we smile, of course our heart palpitates at the sight of the crowd, of course we love the maiden, that beautiful soul clad in that perfect form, so palpably and flagrantly made from all eternity to be loved!”

William James, The Principles of Psychology, vol. 2, New York, 1890, pp. 386-387

Show me a republic, ancient or modern, in which there have been no decorations. Some people call them baubles. Well, it is by such baubles that one leads men.

Napoleon who would have approved of gamification.

5Desrtopa
I don't understand why he specifies "republic" when as far as I can see this properly applies to every society ever. Possibly given the political climate of the time he was using it in a more general way?
6Azathoth123
Probably because he was responding to people who were arguing that these things were inappropriate in a republic.
5Richard_Kennaway
Because he was the leader (not yet Emperor) of a republic founded on the ideals of liberty, equality, and fraternity, and answering egalitarian objections to his creation of the Legion of Honour, not writing a treatise on forms and principles of government.
5Toggle
I suspect that it helped to signal his status as a member of the philosophical tradition of the Enlightenment, and of the French Revolution that was framed in Enlightenment terms. To the extent that the Enlightenment was focused on experimentation with new forms of government, its adherents tended to favor the republic structurally. (Not universally, of course). He also pushed through a number of reforms with the same flavor, and his status in that group seems to have accounted for at least some of his popularity in France. There may also be a subtle religious dimension here. Recall that the Enlightenment entertained Deism and a comparatively radical heterodoxy; using the word 'republic' might have evoked pagan Rome, great icons of the Renaissance (i.e. Florence), and his own enlightened France, while neatly skipping over the most notable Christian theocratic/monarchist governments.
[-]Shmi200

Always train your doubt most strongly on those ideas that you really want to be true.

Sean Carroll in his blog post describing why it is a bit premature to declare that Einstein's General Relativity has been experimentally proven to be incomplete, even if it would be very exciting if so.

1Stabilizer
In other words, use the emotional power of doubt to counteract the bias induced by the emotional power of your desire for that theory to be true.

The point is that even the Good Samaritan had to have the money to help, otherwise he too would have had to pass on the other side.

Margaret Thatcher, CPC Lecture.

No matter how dissatisfied people are with the results they are getting, they rarely question their way of trying to get results. When what we are doing is not working, we do not try doing something totally different. Instead, we try harder by doing more of what seems self-evidently the right way to proceed.

  • Deborah Tannen, You Just Don't Understand, p. 186

"One of the most important things in life is what Judge Learned Hand described as 'that ever-gnawing inner doubt as to whether you're right.' If you don't have that, if you think you've got an inside track to absolute truth, you become doctrinaire, humorless and intellectually constipated. The greatest crimes in history have been perpetrated by such religious and political and racial fanatics, from the persecutions of the Inquisition on down to Communist purges and Nazi genocide." -Saul Alinsky

2HalMorris
Thank you for posting that. I like this quote very much, having started thinking about Alinsky when I started noticing references to his supposed "8 levels of control" for turning the world into a totalitarian zombie factory, beginning with "Control healthcare and you control the people". Does this sound like the same Saul Alinsky? No - it's a myth that started circulating in 2013, and I showed just how far it is from the truth (and how widely it is circulated) in http://therealtruthproject.blogspot.com/2014/07/myths-about-saul-alinsky-and-obama.html So why is Alinsky getting this "Protocols of the Elders of Zion" treatment? I think because this was the year leading up to full "Obamacare" implementation, and another myth has long been established that Alinsky is Obama's Idol (which I think is almost as bogus as the "8 levels of control" but that is far more difficult to establish -- it's what led me to wonder about this: http://lesswrong.com/lw/kfy/an_even_more_modest_search_engine_proposal/
1EGarrett
Well, I feel like politics tends to create bad feelings when it's argued...we'd probably have to have some kind of quarantined thread or pre-agreement to be extra polite if we talked about those topics. But I do like this quote by Alinsky because it's a good habit for everyone on both sides.

What is the first business of one who practices philosophy? To get rid of self-conceit. For it is impossible for anyone to begin to learn that which he thinks he already knows.

-- Epictetus, Discourses

What is the first business of one who practices philosophy?

Publish!

It wasn’t easier, the ghost explains, you just knew how to do it. Sometimes the easiest method you know is the hardest method there is.

It’s like… to someone who only knows how to dig with a spoon, the notion of digging something as large as a trench will terrify them. All they know are spoons, so as far as they’re concerned, digging is simply difficult. The only way they can imagine it getting any easier is if they change – digging with a spoon until they get stronger, faster, and tougher. And the dangerous people, they’ll actually try this.

-Aggy, from Prequel.

On the importance of looking for more efficient ways to do things.

5Sabiola
What is dangerous about that?

The part after it was about how bad guys tend to be like people who have overspecialized in a less useful skill. You will never be able to beat them at what they do, but you don't need to. Said in the context of a very under-powered protagonist. Time for the rest of the quote, though it makes less and less sense as time goes on.

Everyone who will ever oppose you in life is a crazy, burly dude with a spoon, and you will never be able to outspoon them. Even the powerful people, they’re just spooning harder and more vigorously than everyone else, like hungry orphan children eating soup. Except the soup is power. I’ll level with you here: I have completely lost track of this analogy.

I don't know the original context, but I see several possibilities:

  • If the trench really needs to get dug, and it looks like it's going to take digging night and day, then they won't care if they're standing on your toes because stepping off would distract from digging.
  • Similarly, they may conclude everyone needs to be conscripted into the spoon-gangs, including the infirm who will die there and the nerd who was about to invent shovels.
  • If they devote the time and energy to develop their spoon skills they're likely to expect public deference commensurate to their sacrifice, and may get angry when they don't get it.
  • If they do get that deference, and then shovels are developed, they may try to suppress shovels to protect their status.
[-]CCC100

Imagine that the only way you could dig a trench was with a spoon. Imagine that you'd done that - that you'd got stronger, faster, tougher until you, digging with your spoon, could dig a deep trench several metres long.

Now imagine someone gives you a spade. You'd probably be able to divert a fairly large river.

Now imagine someone gives you a spade.

I'd probably call it unethical and try to get it banned.

I'm not sure it plays out this way in real life all that often. For example, anyone who got a digital photography degree before the year 2002 spent three years learning how to accomplish what anyone with a knock-off copy of Photoshop can learn to do in half an hour. They're not super-badass, they're just obsolete.

4RolfAndreassen
To complete your argument, you need to demonstrate that what the man with a degree can do with Photoshop is not super awesome. Maybe the skills transfer; maybe not. But at any rate you haven't demonstrated, or even argued that, they don't.
7sixes_and_sevens
I don't think I need to do that at all. Demonstrating equivalence seems entirely sufficient. EDIT: Wait, I totally misread your comment the first time round, and yes, from the position you're approaching it, the argument isn't complete. In the context of the original spoon/spade argument, I think I'd need to demonstrate that the person with the pre-2002 photography degree can't accomplish equivalent feats to a person with a contemporary photography degree, adjusting for experience. In this case, it seems that the contemporary photography graduate has the advantage of not having to learn a large quantity of material that is no longer necessary with the advent of better tools. This points to a possible difference between badass spoon-diggers and obsolete spoon-diggers: the badass spoon-digger develops general skills, whereas the obsolete spoon-digger develops obsolescent skills. EDIT EDIT: Some explanation on this particular example -- this is a complaint I've heard a couple of times from people with such degrees, that they spent a lot of time learning how to achieve effects which are trivial under contemporary methods. From their perspective, this was time wasted.
5CCC
At the very least, the man with the digital photography degree knows what makes a really good picture - how to fiddle with composition and lighting and stuff to make the sort of picture that makes the viewer go 'wow'. Given a little bit of time to learn how to use Photoshop, or the GIMP, or other similar tools, and the man with the degree will be able to do at least the same as he could do before, but substantially faster. While anyone with a knock-off copy of Photoshop might be able to do the same technical tricks, he probably won't know which technical tricks to use to best improve a given photograph; and he might very well end up making it look a good deal worse.

"My digital photography degree from an era of obsolescent technology isn't rendered completely useless through the passage of time" is a far cry from "I can divert the course of rivers".

6CCC
That is true. This implies that new tools can be divided into two categories; those that use the same skills as the old tools but have a multiplicative effect on the results for all users (spoons->spades), and those that bring even an untrained user up to a certain basic level of competancy but make little or no difference to an expert user. ...having said that, I now realise that there is a third category of tool as well; that which can be used to great effect by an expert but is next to useless to a completely untrained user. An example of this sort of tool would be a bicycle; someone who knows how to ride a bicycle can use it to travel at great speed, while someone who does not know how to use it is likely to simply fall off.
[-]Shmi140

“Nerds are the second scariest group that humanity’s ever produced.”

“Second scariest? Who are the scariest?”

“Stupid people,”

Wildbow of the Worm fame, in Pact

In times like these I really have to wonder why it's never (or at least rarely, to my eye) stressed that self-awareness is probably the single most important component of a healthy life. We're constantly handed very specific definitions of good behavior, complex moral and legal codes, questionable social constructs, and so on. I don't remember ever really being told to take a step back--to step back as far as possible--and look constructively at myself. But increasingly I feel that the only dividing line between being "that guy" and being a net positive to those around you comes out of being able to look at yourself critically and build constructively.

Maybe I'm oversimplifying or assuming that introspection is simple. But for every ten groups explaining religious ideology to me, or telling me why their political candidate is best, I wish one would have told me to get out of my own head as much as possible.

Perhaps we should do what Aristotle meant to do, instead of what he did. The goal he announces in the Metaphysics seems one worth pursuing: to discover the most general truths. That sounds good. But instead of trying to discover them because they're useless, let's try to discover them because they're useful.

-- Paul Graham

The duty of the man who investigates the writings of scientists, if learning the truth is his goal, is to make himself an enemy of all that he reads, and,.. attack it from every side. He should also suspect himself as he performs his critical examination of it, so that he may avoid falling into either prejudice or leniency.

Abū ʿAlī al-Ḥasan ibn al-Ḥasan ibn al-Haytham

9TheMajor
This seems false - the quote illustrates that one should not believe everything just because it was written by a scientist, but there is also such a thing as being too skeptical. With a bit of effort you can deny pretty much any conclusion you dislike. Behaving as advised in this quote is better than taking scientists' word on blind faith, but in order to actually learn the truth from scientific writing you need to invest a lot more effort than just this!
5Richard_Kennaway
I thought this sounded familiar (although that took a bit of work to find -- I wish the site had a better search facility than a Google form).

There's no "should" or "should not" when it comes to having feelings. They're part of who we are and their origins are beyond our control. When we can believe that, we may find it easier to make constructive choices about what to do with those feelings.

— Fred Rogers, The World According to Mister Rogers: Important Things to Remember

5Shmi
So far so good. Indeed. This gets somewhat misleading. While the origins are, the feelings themselves are not. A missing option here is to deconstruct the lightning-fast reasoning steps from the "origins" (e.g. "he downvoted me!") to feelings (e.g. "I am upset!") and see where this chain can be broken and reforged into something more desirable (e.g. ...?). In a more engineering way of describing it, it is not only possible to close the feedback from feelings to "origins", but also to break the feedforward from origins to feelings. The feedforward part is largely intuitive and is harder to analyze, but in some cases it is much more useful.

HEALY: The doctor recommends a bilateral salpingo-oophorectomy.
ROSA: Who doesn't love a surgery with "ooph" in it?
HEALY: Yeah, well, uh the, uh, DOC has set certain limits on invasive... It's not gonna happen. [pause] You're not out of options. We'll stick with the chemo.
ROSA: "We"? You got cancer in your ovaries, too?
HEALY: I'm your counselor. I'm here to help you through this.
ROSA: There is no "through this". I'm gonna die.
HEALY: Hey. Come on, now. You could live for years.
ROSA: That's a fucking lie.
HEALY: Language! Look, I

... (read more)

Berlin wrote: “The dilemma of morally sensitive, honest, and intellectually responsible men at a time of acute polarization of opinion has, since [Turgenev’s] time, grown acute and world-wide.” Whatever Berlin intended, a sentence like this encourages readers to count themselves among the sensitive, honest, and responsible, with the inevitable effect of blinding themselves to their own insensitivities, dishonesties, and irresponsibilities, and to the evils committed by a group, party, or nation that they support. Their “dilemma” is softened by the comforting thought of their merits.

--- Edward Mendelson

4hairyfigment
Note that Berlin was largely right, at a time when many were wrong. But I would say the same about Marxists and Objectivists, at least when it comes to one or two narrow questions, and I think this gave both groups an inflated notion of their own intelligence/rationality.

Very easy to do by default if you haven't done enough historical reading -- especially of primary source -- when you don't realize that what comes natural to you is not the nature of the universe.

Mary Cateli

False opinions are like false money, struck first of all by guilty men and thereafter circulated by honest people who perpetuate the crime without knowing what they are doing.

Joseph de Maistre, Les soirées de Saint-Pétersbourg

Among the worst of barbarisms is that of introducing symbols which are quite new in mathematical, but perfectly understood in common, language. Writers have borrowed from the Germans the abbreviation n! to signify 1.2.3…(n-1).n, which gives their pages the appearance of expressing surprise and admiration that 2, 3, 4 &c. should be found in mathematical results.

Augustus De Morgan, The Penny Cyclopædia of the Society for the Diffusion of Useful Knowledge, London, 1842, vol. 23, p. 444

EDIT: Do the downvotes come from people not getting the fact that this was supposed to be humorous, or from people not sharing my sense of humor?

What's worse is the converse -- where common language users attempt to redefine a precise term for their own purposes. Mathematicians aren't confused by 'perfect numbers,' but I don't even know anymore what people mean around here when they use the B word. Maybe nothing interesting?

2Vulture
Okay, this is probably a stupid question but: What's the B word?
4IlyaShpitser
"Bayesian."
1dthunt
Care to share some hypothetical examples of irritating uses of the B word?
0Vulture
Ah. Of course.

Shall we then call groups, gillygangs, and categories, fenhews?

6Stabilizer
-Philosophy Bro
1lmm
I did not suspect humour; if I had then I would have downvoted.

I have long wondered whether civilization was a mistake. If it was, it is not an easy mistake to avoid. The stubborn persistence of the Comanches aside, once civilized people with technology and professional armies start competing with less civilized people, the results are always going to be lopsided in civilization's favor.

We might be at the bottom of a prisoner's dilemma, the descendants of people who defected from a happy equilibrium of hunting and gathering in order to gain a slight numerical and military advantage over their foes, only to end up wi

... (read more)
3soreff
The empirical evidence that is in the link from Gunnar_Zarncke's post is: This is not just from introspection.

Do you think there would be interest in an "Irrationality Quotes" Thread?

To be honest, these threads are full of such great information that I can't help imagining putting something absolutely useless or ridiculous in it. I just can't resist how it would look to be scrolling through such properly-formatted and thoughtful knowledge from reputable people and then come across, just as perfectly-formatted, presented totally seriously...something like "Some dogs can't resist a tasty morsel of feces." -Theresa A. Fuess. (http://vetmed.illinoi... (read more)

7Alicorn
Similar ideas have been posted before.
1EGarrett
That's funny, I searched for "Irrationality Quotes" and that led me to believe it hadn't. EDIT: Actually now that I think about this...Alicorn may of course not even be suggesting that it's a bad idea due to a previous thread in 2011, but just in general, if the last time the topic came up was 3 years ago, it's probably fine or even preferable to bring it up again in some form if people found it interesting before. I'm sure there have been a few new members in the last 3 years, and a few more irrational things have been said. In general, people discuss various relevant topics more than once, and a time horizon for repeating them would probably reasonably be on the scale of months at the most. Otherwise, it's guaranteeing the slow death of the site.
4Alicorn
I have no particular opinion on whether you ought to start your thread. The links were just FYI because I thought I remembered what might generously be termed prior art on the subject.
4polymathwannabe
Whenever I feel like reading nonsense, I browse the Conservapedia news section.

Experience by itself teaches nothing... Without theory, experience has no meaning. Without theory, one has no questions to ask. Hence, without theory, there is no learning.

-- William Edwards Deming, The New Economics for Industry, Government, Education

5wedrifid
This is false. It is false in theory and it is false in practice. Learning can occur without theory. I spent years researching and developing systems to do just that. And on the practical side (actually human psychology) learning frequently---even predominately---occurs without theory. Abstract theoretical reasoning is a special case of 'learning' and one that is comparatively recent and under-developed in the observed universe.
3khafra
If you're talking about unsupervised classification algorithms, don't they kinda make their theory as they learn? At least, in the "model," or "lossy compression" sense of "theory." Finding features that cluster well in a data set is forming a theory about that data set.

You still need a theory, a.k.a., a prior on the kind of data you expect to be compressing. Otherwise you run into the No Free Lunch Theorem.

-1TheAncientGeek
Theory answers the question "what should I observe"am, .ir rather it answers it internally. On the absence of theory, a learning entity needs to spoon fed data.
0[anonymous]
For example, "having eyes" answers that question, having eyes isn't a theory.
-1AshwinV
Theory is still present. Just because it is not explicitly stated does not mean there is no theory. Abstract theoretical reasoning is learning. The distinction that you are drawing, is just between well stated theory and non-stated implicit theory. (Actually, I suppose it could be viewed as more of a sliding scale depending on how well its explained, how obvious the inferences are etc.,)
5wedrifid
Theories need not be explicit but the lack of explicit theory does not imply that an implicit theory exists or that learning does not occur. One could redefine the term 'learning' such that this claim (and the quote) is tautological. Doing so would be a terrible way to carve reality. No it isn't. The world isn't that neat. Sometimes the theory just isn't there. It would be possible to create a theory that approximates the physical process. But that doesn't mean it exists.
1AshwinV
Can you tell me why it is bad way to carve reality?
1wedrifid
For the same reason it is not helpful to redefine carburetters as 'car stuff'. Abstract theoretical reasoning is different from kinds of learning without theory. It's a remarkable, versatile development that is possessed by comparatively few of the learning systems that are known to exist. That's not a difference to ignore.
2AshwinV
Right. So the difference is probably a technical one.Actually, I'm not sure that I've fully understood your point. Nevertheless, I dont think Edward Deming was talking about the same thing you are. The basic point that he was trying to make is that you need to have some kind of mental model in the light of which you need to analyse your experience.
-4VAuroch
I'm guessing that's about the limits of your knowledge of 'car stuff'. Mostly because the word is 'carburetor'. :-)
4wedrifid
No (read the first sentence. Also, 'Aluminium', 'favour', 'kilometers', ice melts at 0 degrees (or maybe 273.15, but definitely not 32) and a sane incarceration rate per 100,000 population is well under 200 in a stable, established democracy.
3A1987dM
Surely you mean kilometres? :-)
5wedrifid
Damn, yes. I also spell "colour" incorrectly most of the time, ever since I discovered that I could change Nibbles and Gorillas on my 286 by fiddling with the blue and white words.
1VAuroch
Huh. I thought I was familiar with all the American English spelling distinctions. (I checked that Google autocorrected it anyway, but apparently it was too smart for me.) My mistake.
1wedrifid
I've never had cause to write the term down myself. If I recall, I copy and pasted it directly from that wikipedia article, choosing which of the excessive number of location appropriate spelling variants was most aesthetically pleasing to me at the time. Incidentally, I don't have an excessive amounts of practical car knowledge. I've had some interest in the theory behind four stroke internal combustion engines but the specific quirks of the parts that can be seen when looking under the hood of a specific vehicle bore me senseless. But I do know quite a lot about "Cams", my interest being piqued by linguistic affiliation and the desire to understand what on earth the "Camshaft" references my older team-mates used as apparently benign encouragement.
4mwengler
Dogs and humans both learn to walk, run, and jump well through experience, by doing them. To me it is implausible that a one year old human has a theory of walking, running, and jumping which she is using to gain expertise from the experience. But as unlikely as it seems to me that a one year old human has a theory, it seems even less likely that a few-weeks old puppy dog has a theory that she is using to gain expertise from experience. I think there is a kind of learning which needs a theory to happen, but I think that is just one kind of learning.
2bigjeff5
At this point you have to ask what you mean by "theory" and "learning". The original method of learning was "those that did it right didn't die" - i.e. natural selection. Those that didn't die have a pattern of behavior (thanks to a random mutation) that didn't exist in previous generations, which makes them more successful gene spreaders, which passes that information on to future generations. There is nothing in there that requires one to ask any questions at all. However, considering that there is information gained based on past experience, I think the definition of learning could be stretched to cover it. Obviously there is no individual learning, but there is definitely species learning going on there. Since the vast majority of creatures that use this method of learning as their primary method of learning don't even have brains, it seems obvious that there is no theory there. However, if we stretch the definition of theory to include any pattern of information that attempts to reflect reality (regardless of how well it does that job), well then even the lowliest bacteria have theories about how their world is supposed to work, and act accordingly. That same broader definition of "theory" would cover wedrifid's theoryless algorithms as well, as all you care about are patterns of information attempting to reflect reality, and they certainly have those. All that said, the point of the quote is that in order for you as an individual to learn, then you as an individual must have an underlying theory of how things are supposed to be that can be challenged when faced with reality, in order to learn. I have no idea if it's actually true, I'm no psychologist or human learning expert or anything even remotely related, but it sounds like it has to be true even under the strict sense. It seems like it's practically a tautology to me. Even wedfrid's algorithms have a starting framework that attempts to reflect reality, however simplistic it may be. The algorithm its
4Lumifer
Which makes it a practically useless observation, doesn't it?
1bigjeff5
Pretty much.
3mwengler
At a certain point your arguments become circular. If you define learning as something that requires a theory, and then you "find" unstated theories wherever you find learning...
2Lumifer
So let's take theology, for example the medieval catholic one. There certainly was a lot of abstract theoretical reasoning there. Was it learning?
1AshwinV
Hahahaha.. No, it wasnt learning (at least not learning about the real world). Sorry, the referred statement in your comment was a poorly worded one on my part. The point I was trying to make was that I cant seem to envision learning, without having some theory (even if not well formulated) in your head. There has to be some moving parts (or some , that lead to a difference in anticipated outcomes and that should be enough to be fairly called a theory. It has however since been pointed out to me that I might be making a tautological definition of the word "theory". That may be so, and I'm kind of waiting for an explanation as to why that could be a bad idea. (I am of the belief that the more you identify the theory behind a phenomenon, the easier it would be to understand it and work around it/with it.
[-]Shmi30

Quantum probabilities are really credences — statements about the best degree of belief we can assign in conditions of uncertainty — rather than statements about truly stochastic dynamics or frequencies in the limit of an infinite number of outcomes.

Sean Carroll in his blog post about the Born rule in MWI.

0EHeller
If you haven't read the paper talked about in the blog post, you should. It doesn't quite do as much as claimed (fully resolve the problems with Born probabilities in Everett quantum), but it is much more clearly argued than the usual forays into Everett (I suspect its the collaboration between the physicist and the philosopher responsible). In particular, its the cleanest set of assumptions I've seen that lead to a nice Born rule in an Everett situation (Deutsch/Wallace was all sorts of nonsense).
2Shmi
Last impressive bit of progress in the QM fundamentals I recall was Zurek's einselection, where he claimed to show that only the copies of eigenstates survive decoherence. The approach was unusual enough to be taken seriously, even if the experimental confirmation leaves much to be desired. Carroll's "rational observers must believe in Born rule" looks rather dubious to me, mostly because it still takes a classical observer as a fundamental entity.
3pragmatist
Observers are treated as explanatorily fundamental, sure, just as they are in any anthropic-type explanation. But I don't see why that's a problem. The issue is when observers are treated as ontologically fundamental, as they are in some objective collapse interpretations, because that conflicts with the apparent fact that observers are entirely made up out of quantum-mechanical parts. Carroll's paper faces no such conflict.
0Stabilizer
What pragmatist said. Basically the approach of Sebens and Carroll is to show that if observers are present, then they will see outcomes following the Born rule. In that sense it seems that observers here are no more problematic than the observers of special relativity, where there are claims like if you use clocks to measure time in a moving frame, then you will see time slowing down relative to mine.
0EHeller
The piece that is new is showing that only equal amplitude observers can be thought of as "equivalent" for their definition of equivalent. Previously, one "obvious" thing to try to do was to weight different "worlds" equally, and this answers that objection. But also important, its actually pretty clear. A lot of quantum foundations papers are rather poor, its nice to see a contribution thats well reasoned. Edit to add: Just because I think its interesting and clear doesn't mean its correct. In particular, by neglecting the small off diagonal terms in the density matrix they are doing exactly what Everett did in his original derivation, which is only valid in the limit of infinite interactions.

"The history of mathematics is a history of horrendously difficult problems being solved by young people too ignorant to know that they were impossible." -Freeman Dyson

7satt
Duplicate. (In fairness, I only discovered that because I tracked down the original source to try finding out which specific "young people" Dyson had in mind. He seems to imply Abram Samoilovitch Besicovitch as one example; I can put forward George Dantzig as another. But other than them...)
5EGarrett
Apologies for the dupe. Another example that I think is relevant, Wiles decided to solve Fermat's Last Theorem when he was 10-years-old...and picked the problem up again in his early 30's because of that childhood fascination.

There's a saying that goes "People who live in glass houses shouldn't throw stones." Okay. How about "Nobody should throw stones." That's crappy behavior. My policy is: "No stone throwing regardless of housing situation." Don't do it. There is one exception though. If you're trapped in a glass house, and you have a stone, then throw it. What are you, an idiot? So maybe it's "Only people in glass houses should throw stones, provided they are trapped in the house with a stone." It's a little longer, but yeah.

---Demetri Martin, Person (2007)

5wedrifid
Clever reasoning that completely misses the point. Throwing stones is an entirely appropriate response to many situations. It is ill advised (and contemptible) in some others.
1Roxolan
If you're trapped in a glass house and you have a stone, throwing it is still a terrible idea.
5VAuroch
And your counter-proposal for untrapping yourself is? A stone at least breaks the glass at range, which lets you avoid the shards of glass from the initial break, and can be thrown from close enough distance that you can run through the broken area before the entire house starts collapsing, if it was a load-bearing window.
7Roxolan
You'd keep it in your hand and use it as an improvised hammer to carefully break yourself a big enough hole. Hopefully without collapsing the whole house.
[-][anonymous]00

The benefit of knowledge is that it makes the world more predictable, but the cost is that a predictable world sometimes seems less delicious, less exciting, less poignant.

Timothy Wilson, David Centerbar, Deborah Kermer & Daniel Gilbert, ‘The Pleasures of Uncertainty: Prolonging Positive Moods in Ways People Do Not Anticipate’, Journal of Personality and Social Psychology, vol. 88, no. 1 (2005), p. 5

[This comment is no longer endorsed by its author]Reply

One of these days in your travels, a guy is going to come to you and show you a nice, brand new deck of cards on which (Sky snaps fingers) the seal has not yet been broken. This man is going to offer to bet you that he can make the jack of spades jump out of that deck and squirt cider in your ear. Now son, you do not take this bet, for as sure as you stand there, you are going to wind up with an earful of cider."

2VAuroch
Oops. Wasn't active at that time, and didn't see it on a quick search.
7Nomad
I dunno. If the bet is for less money than a can of cider costs and you have a glass ready it might be worth it.

I don't think I would drink cider that had been in my ear. Maybe you have cleaner ears.

That's a side effect of perfecting human rationality. :D

4philh
If it did, they wouldn't make the bet with you. On the other hand, I'd happily pay for the experience of losing that bet.
[-]Shmi-30

Chomsky’s response to a given international event is one of the most predictable phenomena I can think of—even the comets and the tides throw more curveballs. One could easily replace him with a chatbot.

Scott Aaronson.

EDIT: to clarify, if you can predict what a famous personality is going to say on a given topic well enough to replace it with a chatbot, listening to said personality on that topic no longer has much value.

4William_Quixote
Physicists's responses to a given claim of discovering FTL signaling are one of the most predictable phenomena I can think of—even the comets and the tides throw more curveballs. One could easily replace them with a chatbot.
6Wes_W
Well, yeah. If I were a physicist, I might find it annoying to give the same press interview for every individual incorrect claim of FTL signaling. It might be nice if somebody replaced me with a chatbot and let me go back to doing physics. Predictable isn't necessarily the same as wrong. I suppose, here, one must distinguish between listening as "seeking out the person's opinion" and listening as "assigning credence to that opinion". I can listen-1 to crackpots, but I don't listen-2 to them. I listen-2 to my role models, even when I don't need to listen-1 to them (thanks to my ability to model their response).
1Will_Sawin
Do you often read physicist's response to claims of FTL signalling? It seems to me like there is not much value in reading these, per the quote.
4Thomas
Not true. A chatbot (complex enough), can give you an interesting result you haven't thought about it before.
2ChristianKl
I think Scott failed the inverse turing test. When reading his blockquote I would be quite confident in identifying that as something that Chomsky didn't write.
0V_V
Hardly surpising, since his comment was parody.
2private_messaging
Valid moral judgement of "thousand eyes for an eye" actions is inherently pretty simple; if Scott is looking for something very sophisticated, he's not interested in morality. (One may have a sophisticated response to an eye for an eye situation, but a thousand eyes for an eye is pretty one sided when you do not feel affiliated with either group).
-2Jiro
Comparing 1 of something to 1000 of exactly the same thing done under exactly the same circumstances is pretty simple. Needless to say, that doesn't hold true in real life situations. What makes you think it's so simple here?
0private_messaging
What's your argument? That there may be 1000 to 1 difference in the value of life or something? Apples don't have to weight exactly the same as oranges for 1000 apples to be heavier than 1 orange.
8satt
Less Wrong's first Israel/Palestine flame war...this place finally feels like a real blog. [Sniffs, wipes tear from eye.]
-8Jiro
2MondSemmel
Downvoted for the original context of the quote: blue and green politics, strawmanning, etc.
-4Shmi
If you read the comment section, Scott is very careful to avoid blue vs green, as evidenced by the nearly equal split of haters and few supporters (distribution of political views on a given issue is rarely normal, it is bimodal more often than not, so the moderates tend to get more hate than support). Granted, the comment itself is obviously a mockery of Chomsky. The context is that Chomsky is so far down one side, his response is predictable enough to be automated.
7A1987dM
(After that post in Discussion, every time I read “blue” and “green” my brain automatically replaces them with “attractive” and “creepy”.)
3ChristianKl
That's only true if the person who's reading the post can only see black and white and who doesn't care about deeper understanding and arguments. In this case Chomsky would probably speak about how Hamas usually does claim responsibility for terror acts that it commits. The latest conflict escalated on grounds that Israel claimed that Hamas is responsible for a kidnapping for which Hamas didn't claim responsibility. Speaking about the timing and the motivation of various parties for the kidnapping could be interesting. Analysis of Egypt's role in this conflict would be interesting. Especially as Egypt lately outlawed the Muslim brotherhood.
1V_V
So it is aqua politics?
0V_V
I've heard that D-Wave quantum computer solves the P vs NP problem. I wonder what Scott Aaronson has to say about it. :D
0Shmi
Good point.
0wedrifid
This is true to the extent that all of one's beliefs, thoughts and feelings are reflectively consistent, incorporating all available evidence into a cohesive, integrated whole. Comparatively little of the process of learning is the absorbing of evidence not previously heard. Most of the benefits come from deliberate practice#Deliberate_practice) and repetition. If I didn't listen to the advice of people whose advice generation I could emulate I would be much worse off.
0satt
Having clicked through and read the rest of Scott's comment, I feel compelled to add the proviso that when applying this heuristic, one should check whether one's predictions are, in fact, accurate.