= 27d567bc2d48d9f40e9ccc20ea370b15)
Rationality quotes: April 2010
This is our monthly thread for collecting these little gems and pearls of wisdom, rationality-related quotes you've seen recently, or had stored in your quotesfile for ages, and which might be handy to link to in one of our discussions.
- Please post all quotes separately, so that they can be voted up/down separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)
- Do not quote yourself.
- Do not quote comments/posts on LW/OB.
- No more than 5 quotes per person per monthly thread, please.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
Comments (307)
True Knowledge:
Life is a process of breaking down and using other matter, and if need be, other life. Therefore, life is aggression, and successful life is successful aggression. Life is the scum of matter, and people are the scum of life. There is nothing but matter, forces, space and time, which together make power. Nothing matters, except what matters to you. Might makes right, and power makes freedom. You are free to do whatever is in your power, and if you want to survive and thrive you had better do whatever is in your interests. If your interests conflict with those of others, let the others pit their power against yours, everyone for theirselves. If your interests coincide with those of others, let them work together with you, and against the rest. We are what we eat, and we eat everything.
All that you really value, and the goodness and truth and beauty of life, have their roots in this apparently barren soil.
This is the true knowledge.
We had founded our idealism on the most nihilistic implications of science, our socialism on crass self-interest, our peace on our capacity for mutual destruction, and our liberty on determinism. We had replaced morality with convention, bravery with safety, frugality with plenty, philosophy with science, stoicism with anesthetics and piety with immortality. The universal acid of the true knowledge had burned away a world of words, and exposed a universe of things.
Things we could use.
--Ken MacLeod, The Cassini Division
'True Knowledge'? Only if you include the capital 'T' and 'K'!
This is not 'the universal acid of the true knowledge burning away a world of words'. It's just a world of words.
Ceci n'est pas une pipe.
It disappoints me that this kind of thing is still news to some people. I value survival (that is, the continued existence of things similar to myself) first and foremost, partly because it's the one thing my ancestors have had in common ever since the invention of phospholipid membranes. The state of Existence is, metaphorically, engaged in ongoing skirmishes with it's various neighbors in possibilty-space, so I'd rather stay away from the border, just to avoid getting caught on the wrong side if it shifts.
Currently, humans don't work that way. I mean, sure, we want to survive, and will do a lot of nasty things for it, but if you actually internalize nihilism, crass self-interest, and convention as your moral foundation, then the result will NOT be goodness or truth or beauty. To win, you have to be aware of the mundane roots of things without celebrating them.
See, e.g., Gall's Law and/or Goodhart's Law.
No, currently we don't. If we want our values to survive, then we must win. If we want to win, we have nothing else to place our values on besides this "apparently barren soil".
Think of it as the converse of the following Terry Pratchett dialog between Susan and Death in Hogfather:
"All right," said Susan. "I'm not stupid. You're saying humans need... fantasies to make life bearable."
"REALLY? AS IF IT WAS SOME KIND OF PINK PILL? NO. HUMANS NEED FANTASY TO BE HUMAN. TO BE THE PLACE WHERE THE FALLING ANGEL MEETS THE RISING APE"
"Tooth fairies? Hogfathers? Little- "
"YES. AS PRACTICE. YOU HAVE TO START OUT LEARNING TO BELIEVE THE LITTLE LIES"
"So we can believe the big ones?"
"YES. JUSTICE. MERCY. DUTY. THAT SORT OF THING"
"They're not the same at all!"
"YOU THINK SO? THEN TAKE THE UNIVERSE AND GRIND IT DOWN TO THE FINEST POWDER AND SIEVE IT THROUGH THE FINEST SIEVE AND THEN SHOW ME ONE ATOM OF JUSTICE, ONE MOLECULE OF MERCY. AND YET-- " Death waved a hand. "AND YET YOU ACT AS IF THERE IS SOME IDEAL ORDER IN THE WORLD, AS IF THERE IS SOME... SOME RIGHTNESS IN THE UNIVERSE BY WHICH IT MAY BE JUDGED"
"Yes, but people have got to believe that, or what's the point---"
"MY POINT EXACTLY"
I enjoyed the Pratchett dialogue, but I am not sure I learned from it -- I wind up empathizing with both characters. Are you agreeing with me? Disagreeing with me? What is the converse of a dialogue? I'm confused.
I think part of what bothers me about your Cassini quote is that the claims in the first paragraph are overstated, especially coming from a character who is (presumably) a metaethical nihilist/egoist.
Why, is it so wrong to eat things? Eating seems normal and natural to me; an activity to be celebrated. "Scum" is a kind of life that prevents our usual foods from being healthy for us -- it is thus an odd insult for a carnivore.
Why? If I firmly estimate that other minds exist, does the existence of those minds depend upon my estimation? If other minds exist, why should what matters to them be irrelevant? What does it even mean to say that "might makes right" except that I plan to ignore the concept of "right"? When, in the course of human events, has the power to ignore morality left people truly free?
Really? All the time? Is the world so grim that I must spend all my time eating or face extinction? Surely species and individuals with a significant advantage can spend some of the resulting surplus on frivolous pursuits; what evidence is there that the fate of the world hangs by a razor-thin thread?
Mastication is only one form of eating. As a Westerner, I consume a large portion of our world's resources in the form of energy, household goods, large appliances, transportation, gadgets, taxes to fund war efforts, etc.
As for imposing our will upon life, just look at factory farms, algae farms, dead zones in the sea, global warming, and war. Might is truly the final arbiter, and unless part of what we care about is the other, then we show a good track record of trampling them for our own uses.
Our present (relative) peace was brought about by people who felt the rights of others mattered, and had the might to back it up and impose it on those who felt differently.
You're supposed to, or at least I did. Both are right.
The converse of a logical statement is another statement with the antecedent and consequent swapped. I was using it metaphorically for "another similar take on the same subject". Both these quotes emphasize that there is no morality inherent in the universe. If we want a moral universe, we have to build it ourselves.
The Cassini Division quote actually to me seems rather cheerful. Even from cynicism that deep we can build a good life full of all the things we cherish.
I think that's because they're not coming from a unitary viewpoint. They're bridging between something approximating normal morality, and utter amorality.
The point is not "it's wrong to eat things". The point is that life is what's survived, and it does anything it can to survive. People much the same, though they're better at it.
Of course not.
First ask why should what matters to them be relevant?
Well, because:
1. You want to live under conditions such that they are.
2. They're useful to you, and you to them, and cooperation can make you both better off than a bitter fight to the death.
But neither of these is fundamental.
It means that the concept of right is not fundamental, is not baked into the fabric of the universe. Right only means something relative to the minds that hold it. And they can only enforce that with might. Try reading it as "might effects right".
Well, the simplest answer is when people have the power to ignore morality forced upon them by others that they don't agree with. If a gay man is free to ignore the moral judgements of an Imam in a Sharia country, he is freer to have sex with whom he pleases, how he pleases. A slave that has the power to escape is freer. A person is freer when they can do something that pleases them rather than the high-paying stressful job that their parents tell them is what they should do.
All the time.
The "frivolous pursuits" are both the thriving and what is in your interests. You interests include both accumulating the surplus and spending it on what matters to you.
The times where it is survival on the line, rather than thriving, can be much rarer.
All right, all of that is interesting. I would use some of the words you use differently, but none of your definitions are unreasonable, and now that I understand what you're really saying, I agree with most of it.
I still disagree that the interests of others are non-fundamental; there are causes I would die for, which your philosophy seems to forbid. Perhaps I still don't understand your stance on that point.
Also, this may be nitpicky, but at this point in history, life is not "what survived." The ocean, the moon, the molten core of the Earth, the Sun, and, so far as we know, much of the rest of the galaxy are made of nonliving matter that is roughly as enduring as life. Life has not yet succeeded in eating everything else.
:-)
You're free to do so, should you decide that's what you value.
It's not my philosophy, or at most only a minor part. I like seeing what this viewpoint illuminates, and thought others here would as well. Judging by the karma swings on the post, it has proven controversial. Hopefully it's provoked some thought in doing so.
Nitpicks are good. That's an entirely fair point. I wavered between this formulation and a statement that life is the only thing that uses other matter, which I think is closer to expressing a violation of the Kantian categorical imperative (second formulation), and hence a common formulation of evil. (Or as Pratchett expressed it: "And sin, young man, is when you treat people as things. Including yourself. That's what sin is.")
Me too!
As long as you're accepting nitpicks, I don't think your Kantian formulation holds much water either. Kant teaches that you can't use people as means to an end, but he would probably encourage you to use things as means to an end; certainly he chides people who want to leave well enough alone (thereby saving resources) for not developing their latent talents.
A hookworm might be evil under a modernish Kantian framework because it is life treating other life as a means to an end; ditto fire, which is matter treating other matter as a means to an end. A lichen, though, is well within its rights (as life) to treat the rock it sits on (matter) as means to an end, and people (as the only known Kantian-rational agents) are well within their rights to treat wheat (mere life) as means to an end.
I have not read The Cassini Division. Are these MacLeod's views? That is, in context, is it written to present the speaker's philosophy as good or evil? MacLeod himself, I believe, is something of a socialist, which would make these views rather odd in his own mouth.
His views are ... eclectic. He has self described as a Libertarian Trotskyist. Don't ask me how that works. I think it would be an exaggeration to say these are MacLeod's views.
They are presented as a very distorted reading of philosophy and politics shaped by the founders' horrible pre-revolution life:
They are also, however, the views of the one of the largest political entities in the solar system, indicating that whatever else, they work (in that fictional universe). They're also the views of the main character. She's designed pretty explicitly to be something of a Rorschach test, entirely ambiguous between a monster and the saviour of the human race.
ROT13: Fur (nggrzcgf gb?) pbzzvgf trabpvqr ntnvafg n cbfguhzna pvivyvmngvba yvivat va bar bs gur tnf tvnagf. Guvf cbfguhzna pvivyvmngvba unf znqr fbzr ntterffvir zbirf ntnvafg uhznavgl, ohg gur erprag barf unir orra ragveryl bs gur sbez bs vasbezngvba jnesner nggnpxf -- gnxvat bire pbzchgref.
I'm puzzled that this has such low karma.
Perhaps many people were turned off by the poetic imagery, but it seemed that many responders failed to understand it. It can be very difficult to step outside the human frame, and truly consider reality as matter, forces, space and time. A lot of the replies seemed to find this statement repugnant, without realizing that it's just an explanation of facts.
Also, for those who think this excerpt is an exhortation to nihilism: read the final paragraph again. We build upon the basics. We form societies because they meet our individual preferences, we establish moral conventions because we desire something from our individual experiences.
I didn't find it engaging, so I didn't bother finishing it when I saw it. Reading the entire thing in reaction to your comment, I don't find it interesting.
Understood.
It may be a basic difference of interests. Less Wrong attracts engineers, economists, scientists, and philosophers, who will find value in different kinds of comments / topics.
I was just surprised, given the reaction to similar material here in the past, that this wasn't heavily upvoted.
As PhilGoetz said, "Peanut butter gets more karma than caviar."
In recognition of your opinion, I will devote further analysis to the quote at a later time. My opinion may be overly shallow.
-- Anne Frank, 3 May 1944, aged 14
Rising in revolt tends to mean civil war. Perhaps if she thought that through a little more she would find at least one answer. One reason to stop other crazy people destroying things you value is to kill them.
Defense is necessary, I agree. But perhaps the revolt she was looking for was one of peaceful protest on both sides. The leaders can't do much damage without followers and supporters, armies and engineers. This site has already covered many of the biases which would lead one to support war, regardless of cause.
The other side seems to agree:
-- Adolf Hitler
Too true!
-- Jack Handey's Deep Thoughts
-- Snocone, in a Slashdot post
-- Bruce Lee
-- George Eliot
-- Christopher Hitchens
Accuracy was sacrificed for a pleasant parallel construction. Anything can be so asserted.
And, without supporting evidence, such assertions demonstrate nothing.
"There are no married bachelors."
My dad has a Bachelor's degree.
Is he married?
Yes, to mom.
"There are no married unmarried men."
I add this grudgingly, as deliberately seeking ambiguity in a clear sentence is just being fatuous; it's not a valid objection.
The mere fact that an assertion has been made is, in fact, evidence. For example, I will now flip a coin five times, and assert that the outcome was THHTT. I will not provide any evidence other than that assertion, but that is sufficient to conclude that your estimate of the probability that it's true should be higher than 1/2^5. Most assertions don't come with evidence provided unless you go looking for it. If nothing else, most assertions have to be unsupported because they're evidence for other things and the process has to bottom out somewhere.
Now, as a matter of policy we should encourage people to provide more evidence for their assertions wherever possible, but that is entirely separate from the questions of what is evidence, what evidence is needed, and what is demonstrated by an assertion having been made.
Or it's sufficient to conclude that one's estimate should be less than 1/2^5. Without providing additional evidence (such as "I saw the THHTT outcome") your claim is rather dubious and -- in the realm of humans -- this probably is a good indicator that you are lying or are crazy. I'm not sure how one should update your posteriors.
Suppose I tell you that my password is D!h98+3(dkE4. Do you conclude that since I don't want you to know my password, I must be trying to mislead you as to what my password is, and so the probability that this is my password is actually less than 1/95^12?
If I assert that the outcome as THHTT, either I'm lying or I'm not lying, and there's little evidence either way. What little evidence there is probably doesn't push my probability of telling the truth below 3%, and surely the strength of the evidence has little, if anything, to do with the prior probability of the coin showing THHTT.
Good point. Thanks for batting down my idiocy here, much obliged =D
Well, clearly we can assert anything we want, so the quote becomes:
And we notice that evidence doesn't change depending on whether you're considering something for belief or dismissal, so the quote becomes:
So Hitchens is really telling us that prior probabilities tend to be small, which is true since there are almost always many possible hypotheses that the probability mass is split between.
You're assuming that probability mass tends to be split between stuff. This would be true, if all interesting statements were mutually exclusive or something. But consider the hypothesis that at least one statement in the Bible is true. This hypothesis is very complex, and yet its prior probability is very large.
One thing that bugs me about this quote is that it isn't strong enough. It might give people the impression that it's up to the reader's opinion or personal preference to decide what to believe or not believe. They're allowed to believe in something they have no evidence for, you're allowed to dismiss it, everyone's happy.
Sir Kenneth Clarke, "Civilisation" (Excerpt on YouTube.)
Richard Bellman, "Eye of the Hurricane"
Sounds like a traditional-rationality precursor to "hypotheses are expectation-constrainers".
It doesn't sound like that to me. Can you elaborate?
Well, the quote could be interpreted as "Any scientific theory must ultimately produce some numbers, so that reality can be measured and we can see whether the numbers match."
Another interpretation is "A scientific theory ultimately isn't a scientific theory at all unless it's essentially a set of equations."
I agree, although I think the second sentence ("Theories stand or fall, ultimately, upon numbers") is sufficient to justify the former interpretation.
Charles Darwin, "The Expressions of the Emotions in Man and Animals", ch.3.
Crap Mariner (Lawrence Simon)
The opposite of rose-tinted spectacles: shit-tinted shades.
Gall's Law:
John Gall, "Systemantics"
Counterexample: Space shuttle.
Evolved from both simpler winged aircraft and simpler rockets.
All the base components that went into the space shuttle still existed on a line of technogical progress from the basic to the advanced. Actually, the space shuttle followed Gall's Law precisely.
The lift mechanism was still vertically stacked chemical rockets of the sort that had already flown for decades. The shuttle unit was built from components perfected by the Gemini and Apollo programs, and packed into an aerodynamic form based on decades of aircraft design.
Reducing technologically, the shuttle still depends on simple systems like airfoils, rockets and nozzles, gears, and other known quantities.
Then if that qualifies, what would falsify Gall's Law?
The first development of the electronic circuit would have been a case of a complex technological system that worked, but was not based fundamentally upon existing simpler machines. The first use of chemical propulsion - gunpowder / rocketry - might have been a similar case.
(EDIT: Upon further consideration, chemical propulsion is based upon the simpler technologies of airtight confinement and incendiary materials. However, I still think the electronic circuit was effectively the rise of a new fundamental device with complex behavior unconnected to more basic technologies. If anyone thinks they can reduce the circuit to simpler working devices I would be fascinated to explore that.)
It's a good question. I'm turning over various possibilities in my mind.
Do you still hold that the space shuttle falsifies it?
If so, I'd be interesting in hearing your reasoning, and other examples you consider similar.
Electroplating and electrolysis of water both involve a circuit, but aren't overwhelmingly complex. Samuel Thomas von Sommering's electrochemical telegraph was based on electrolysis. It's not like someone pulled doped silicon semiconductors straight out of the lightning-struck sand.
True, +1 for a thoughtful answer.
However, I still don't see the circuit as reducible to simpler working components. Regardless of the medium across which the current flows, it still seems to me that the circuit is a simple machine - a basic device like the pulley, joint, inclined plane, or lever.
In considering this, I also think that chemical fuels are simple machines and belong on that list, as they are ostensibly devices (can be used by an agent to do work) but also aren't reducible to simpler working components.
Basically, the shuttle is a system of rockets carrying a space-worthy airplane as payload. Both of these components had predecessors. Had the shuttle been the first rocket or first space-worthy airplane, it would have falsified Gall's Law.
I'm not sure.
Isn't the first rocket or airplane also built on simple technologies?
Couldn't one continue to reduce components to simpler devices until you get to basic joints, inclined planes, tensors (springs), incendiary materials (fuel), etc - that all would have had to be developed and understood before an engineer could design the rocket / airplane?
(EDIT: I realize that I'm essentially positing that Gall's Law holds if all technology should be reducible to simple machines, and that what we call "technology" is improving, refining, and combining those designs.)
I'm not saying that the first rocket and first airplane falsified Gall's Law. I'm saying that, had the space shuttle, in the form in which it was actually built, been the first rocket or the first airplane, it would have falsified Gall's Law.
Ah, I understand.
Total agreement.
Suppose a hyperintelligent alien race did build a space shuttle equivalent as their first space-capable craft, and then went on to build interplanetary and interstellar craft.
Alien 1: The [interstellar craft, driven by multiple methods of propulsion and myriad components] disproves Gall's Law.
Alien 2: Not at all. [Craft] is a simple extension of well-developed principles like the space shuttle and the light sail.
You can simply define a "working simple system" as whatever you can make work, making that a pure tautology.
I agree.
All of these concepts are imprecisely connected to the real world. Does anyone have an idea for how we could more precisely define Gall's Law to more ably discuss real expected experience?
I'm considering a definition which might include the phrase:
"Reducible to previously understood components"
I think the key insight here is that you get a limited number of bits, in design space, to bridge between things that have already been shown to work, and things that have yet to be shown to do so.
For purposes of Gall's law, we are interested in the number of bits of design that went into the space shuttle without ever having been previously shown to work. So you have to subtract off the complexity of "the idea of an airplane", which we already had, and of the solid fuel booster rockets, which we already knew how to build; and also of any subassembly which got built and tested successfully in a lab first -- but perhaps leaving some bits or fraction of a bit to account for the unknown environment when using them on the real shuttle, versus in the lab.
I would say that Gall's Law is about the design capacities of human beings (like Dunbar's Number), or is something like "there's a threshold to how much new complexity you can design and expect to work", with the amount of complexity being different for humans, superintelligent aliens, chimps, or Mother Nature.
(the limit is particularly low fo Mother Nature - she makes smaller steps, but got to make much more of them)
That's not my point. My point is that Gall's law is unfalsifiable by anything short of Omega converting its entire light cone into computronium/utilium in a single, plank-time step.
Edit: Not to say that Gall's Law can't be useful to keep in mind during engineering design.
Further reply:
I was contemplating this exchange and wondering whether Gall's Law has any value (constrains expected experience).
I think it does. If an engineer today claimed to have successfully designed an Albucierre engine, I would probably execute an algorithm similar to Gall's Law and think:
The technology does not yet exist to warp space to any degree, nor is there an existing power source which could meet the needs of this device. The engineer's claim to have developed a device which can be bound to a craft, controllably warp space, and move it faster than light is beyond existing technological capability. We are too many Gall Steps away for it to be probable.
The Columbia shuttle crew would still be with us if this were correct.
True, the space shuttle was not completely contained on its vertical axis, but I was talking about the boosters themselves. I said the lift mechanism was a vertically stacked chemical rocket, not that the entire shuttle was a uniform tower, as it obviously wasn't.
The boosters are components of the space shuttle, which is what we were talking about: simpler working components evolving into complex systems.
Simple working component: Rocket booster
Complex system: Shuttle with a crew module, fuel tanks, and multiple boosters
In addition to NMJablonski's point, it is perhaps arguable just how well the Space Shuttle worked. In hindsight it seems that the same amount of orbital lift capacity could have been done rather more cheaply.
It works for a job it isn't used for: launching into a polar orbit to emplace secret military satellites, and gliding a very long distance back to base without a need for a splashdown recovery that might risk its secrecy.
That's what gave it the wings, and once you have the wings the rest of the design follows.
It doesn't qualify 100%, because there were little prototype shuttles. Still, you have a point. If we have good theories, we can build pretty big systems from scratch. Gall's law resonates especially strongly with programmers because much of programming doesn't have good theories, and large system-building endeavors fail all the time.
Even if there hadn't been prototype shuttles, the shuttle is still reducible to simpler components. Gall Law just articulates that before you can successfully design something like the space shuttle you have to understand how all of its simpler components work.
If an engineer (or even transhuman AI) had sat down and started trying to design the space shuttle, without knowledge of rocketry, aerodynamics, circuits, springs, or screws, it would be pulling from a poorly constrained section of the space of possible designs, and is unlikely to get something that works.
The way this problem is solved is to work backwards until you get to simple components. The shuttle designer realizes his shuttle will need wings, so starts to design the wing, realizes the wing has a materials requirement, so starts to develop the material. He continues to work back until he gets to the screws and rivets that hold the wing together, and other simple machines.
In engineering, once you place the first atom in your design, you have already made a choice about atomic mass and charge. Complex patterns of atoms like space shuttles will include many subdivisions (components) that must be designed, and Gall's Law illustrates that they must be designed and understood before the designer has a decent chance of the space shuttle working.
I think you completely miss the point of Gall's law. It's not about understanding individual components. Big software projects still fail, even though we understand if-statements and for-loops pretty well.
I know that.
It's about an evolution from simpler systems to more complex systems. Various design phases of the space shuttle aren't what falsify that example. It's the evolution of rocket propulsion, aircraft, and spacecraft, and their components.
(EDIT: Also, at no point was I suggesting that understanding of components guarantees success in designing complex systems, but that it is neccessary. For a complex system to work it must have all working components, reduced down to the level of simple machines. Big software projects would certainly fail if the engineers didn't have knowledge of if-statements and for-loops.)
Really? I think only 6 of them were built, and 2 of those suffered catastrophic failure with all hands lost.
The "inverse proposition" given is actually the contrapositive of (i.e. is equivalent to) the original statement.
Counterexample: a complex computer program designed and written from scratch.
...and that worked the very first time? How often does that happen?
The quote is a rule of thumb and an admonition to rational humility, not a law of the universe.
I agree it's probably not a law of the universe, as I cannot rule out possible minds that could falsify it. However, I cannot from within my mind (human capabilities) see a case where a complex system could work before each of its parts had been made to work.
Well "never works and cannot be made to work" does sound a bit strong to me.
I've written some of those. And every time, I test everything I write as I go, so that at every stage from the word go I have a working program. The big bang method, of writing everything first, then running it, never works.
The "big bang" sometimes happens to me when I write in Haskell. After I fix all the compiler errors, of course. I just wish there were a language with a type system that can detect almost as many errors as Haskell's without having quite such a restrictive, bondage-fetish feel to it.
But yeah, in general, only trivial programs work the first time you run them. That's a good definition of trivial, actually.
Robert Pirsig, Zen and the Art of Motorcycle Maintenance
Daniel Dennett, interview for TPM: The Philosopher's Magazine
Joe Biden, remarks delivered in Saint Clair Shores, MI, Monday, September 15, 2008
Of course, to really see what someone values you'd have to see their budget profile across a wide range of wealth levels.
-- R Scott Bakker, Neuropath
You mean, like every Bayesian believes their prior is correct?
Bayesians don't believe they lucked into their priors. They have a reflectively consistent causal explanation for their priors.
Even if their explanation were correct, they would still have lucked into them. Others have different priors and no doubt different causes for their priors. So those Bayesians would have been lucky, in order to have the causes that would produce correct priors instead of incorrect ones.
But that still doesn't need to be luck. I got my priors offa evolution and they are capable of noticing when something works or doesn't work a hundred times in a row. True, if I had a different prior, I wouldn't care about that either. But even so, that I have this prior is not a question of luck.
Then it would make sense to use some evolutionary thingy instead of Bayesianism as your basic theory of "correct behavior", as Shalizi has half-jokingly suggested.
It is luck in a sense - every way that your opinion differs from someone else, you believe that factors outside of your control (your intelligence, your education, et cetera) have blessed you in such a way that your mind has done better than that poor person's.
It's just that it's not a problem. Lottery winners got richer than everyone else by luck, but that doesn't mean they're deluded in believing that they're rich. But someone who had only weak evidence ze won the lottery should be very skeptical. The real point of this quote is that being much less wrong than average is an improbable state, and you need correspondingly strong evidence to support the possibility. I think many of the people on this site probably do have some of that evidence (things like higher than average IQ scores would be decent signs of higher than normal probability of being right) but it's still something worth worrying about.
I think I agree with that: There's nothing necessarily delusive about believing you got lucky, but it should generally require (at least) an amount of evidence proportional to the amount of purported luck.
Priors can't be correct or incorrect.
(Clarified in detail in this comment.)
They can be more or less useful, though.
According to what criterion? You'd end up comparing a prior to the prior you hold, with the "best" prior for you just being the same as yours. Like with preference. Clearly not the concept Unknowns was assuming -- you don't need luck to satisfy a tautology.
Of being better at predicting what happens, of course.
You can't judge based on info you don't have. Based on what you do have, you can do no better than current prior.
I am in violent agreement.
But you can go and get info, and then judge, and say, "That prior that I held was wrong."
You're speaking as if all truth were relative. I don't know if you mean this, but your comments in this thread imply that there is no such thing as truth.
You've recently had other discussions about values and ethics, and the argument you're making here parallels your position in that argument. You may be trying to keep your believes about values, and about truths in general, in syntactic conformance. But rationally I hope you agree they're different.
Sounds mysterious to me. Priors are not claims about the world?
Not quite. They are the way you process claims about the world. A claim has to come in context of a method for its evaluation, but prior can only be evaluated by comparing it to itself...
This downvoting should be accompanied with discussion. I've answered the objections that were voiced, but naturally I can't refute an incredulous stare.
The normal way of understanding priors is that they are or can be expressed as joint probability distributions, which can be more or less well-calibrated. You're skipping over a lot of inferential steps.
Right. We could talk of quality of an approximation to a fixed object that is defined as the topic of a pursuit, even if we can't choose the fixed object in the process and thus there is no sense in having preferences about its properties.
I can't tell what you're talking about.
Say, you are trying to figure out what the mass on an electron is. As you develop your experimental techniques, there will be better or worse approximate answers along the way. It makes sense to characterize the approximations to the mass you seek to measure as more or less accurate, and characterize someone else's wild guesses about this value as correct or not correct at all.
On the other hand, it doesn't make sense so similarly characterize the actual mass of an electron. The actual mass of an electron can't be correct or incorrect, can't be more or less well-calibrated -- talking this way would indicate a conceptual confusion.
When I talked about prior or preference in the above comments, I meant the actual facts, not particular approximations to those facts, the concepts that we might want to approximate, not approximations. Characterizing these facts as correct or incorrect doesn't make sense for similar reasons.
Furthermore, since they are fixed elements of ideal decision-making algorithm, it doesn't make sense to ascribe preference to them (more or less useful, more or less preferable). This is a bit more subtle than with the example of the mass of an electron, since in that case we had a factual estimation process, and with decision-making we also have a moral estimation process. With factual estimation, the fact that we are approximating isn't itself an approximation, and so can't be more or less accurate. With moral estimation, we are approximating the true value of a decision (event), and the actual value of a decision (event) can't be too high or too low.
Prior can't be judged. It's not assumed to be "correct". It's just the way you happen to process new info and make decisions, and there is no procedure to change the way it is from inside the system.
Locked in, huh? Then I don't want to be a Bayesian.
Since you are already locked in in some preference anyway, you should figure out how to compute within it best (build a FAI).
What makes you say that? It's not true. My preferences have changed many times.
What makes you say that Bayesians are locked in? It's not true. If they're presented with evidence for or against their beliefs, they'll change them.
You're talking about posteriors. They're talking about priors, presumably foundational priors that for some reason aren't posteriors for any computations. An important question is whether such priors exist.
But your beliefs are your posteriors, not your priors. If the only thing that's locked in is your priors, that's not a locking-in at all.
That's not obvious. You'd need to study many specific cases, and see if starting from different priors reliably predicts the final posteriors. There might be no way to "get there from here" for some priors.
When we speak of the values that an organism has, which are analogous to the priors an organism starts with, it's routine to speak of the role of the initial values as locking in a value system. Why do we treat these cases differently?
Distinguish formal preference and likes. Formal preference is like prior: both current beliefs and procedure for updating the beliefs; beliefs change, but not the procedure. Likes are like beliefs: they change all the time, according to formal preference, in response to observations and reflection. Of course, we might consider jumping to a meta level, where the procedure for updating beliefs is itself subject to revision; this doesn't really change the game, you've just named some of the beliefs changing according to fixed prior "object-level priors", and named the process of revising those beliefs according to the fixed prior "process of changing object-level prior".
When formal preference changes, it by definition means that it changed not according to (former) formal preference, that is something undesirable happened. Humans are not able to hold their preference fixed, which means that their preferences do change, what I call "value drift".
You are locked in in some preference in normative sense, not factual. This means that value drift does change your preference, but it is actually desirable (for you) for your formal preference to never change.
I object to your talking about "formal preference" without having a formal definition. Until you invent one, please let's talk about what normal humans mean by "preference" instead.
I'm trying to find a formal understanding of a certain concept, and this concept is not what is normally called "preference", as in "likes". To distinguish from the word "preference", I used the label "formal preference" in the above comment to refer to this concept I don't fully understand. Maybe the adjective "formal" is inappropriate for something I can't formally define, but it's not an option to talk about a different concept, as I'm not interested in a different concept. Hence I'm confused about what you are really suggesting by
For the purposes of FAI, what I'm discussing as "formal preference", which is the same as "morality", is clearly more important than likes.
I'd be willing to bet money that any formalization of "preference" that you invent, short of encoding the whole world into it, will still describe a property that some humans do modify within themselves. So we aren't locked in, but your AIs will be.
If someone was locked in to a belief, then they'd use a point mass prior. All other priors express some uncertainty.
But one man's prior is another man's posterior: I can use the belief that a medical test is 90% specific when using it to determine whether a patient has a disease, but I arrived at my beliefs about that medical test through Bayesian processes - either logical reasoning about the science behind the test, or more likely trying the test on a bunch of people and using statistics to estimate a specificity.
So it may be mathematically wrong to tell me my 90% prior is false, but the 90% prior from the first question is the same 90% posterior from the second question, and it's totally kosher to say that the 90% posterior from the second question is wrong (and by extension, I'm using the "wrong prior")
The whole reflective consistency thing is that you shouldn't have "foundational priors" in the sense that they're not the posterior of anything. Every foundational prior gets checked by how well it accords with other things, and in that sense is sort of a posterior.
So I agree with cousin_it that it would be a problem if every Bayesian believed their prior to be correct (as in - they got the correct posterior yesterday to use as their prior today).
Vladimir is using "prior" to mean a map from streams of observations to probability distributions over streams of future observation, not the prior probability before updating. Follow the link in his comment.
I have heard some argue for adjusting priors as a way of dealing with deductive discoveries since we aren't logically omniscient. I think I like that solution. Realizing you forgot to carry a digit in a previous update isn't exactly new information about the belief. Obviously a perfect Bayesian wouldn't have this issue but I think we can feel free to evaluate priors given that we are so far away from that ideal.
""Not evil, but longing for that which is better, more often directs the steps of the erring"
Theodore Dreiser, Sister Carrie
--Robert A. Heinlein
Sad, but true.
"Hypocrisy and dissimulation are what keeps social systems strong; it is intellectual honesty that destroys them."
Theodore Dalrymple- The New Vichy Syndrome p. 26.
This is true when the social systems in question are built on dishonest foundations. Observing whether or not intellectual honesty has this effect on a system has predictive value wrt the eventual fate of the society employing the system.
Voted up.
If the rationality quotes are intended to illustrate rationality, rather than themselves necessarily be rational, I think this is a fine quote.
Source:
-- Schelling, Strategy of conflict, p144
[The book was mentioned a couple of times here on LW, and is a nice introduction to the use of game theory in geopolitics]
--Voltaire
Wandering in a vast forest at night, I have only a faint light to guide me. A stranger appears and says to me: 'My friend, you should blow out your candle in order to find your way more clearly.' The stranger is a theologian.
But blowing out the candle actually would make it easier to find your way (it ruins your night vision).
Not if the forest is sufficiently dark that your night vision doesn't have enough light to work with.
That seems like an easy case to test, provided you have some way to re-light the candle.
You need to make two assumptions for the analogy.
1) You can't re-light the candle.
2) If you do things exactly right, you'll get out with just before starving to death (or dying somehow) otherwise, you are dead.
Tom Siegfried, Odds Are, It's Wrong, on the many failings of traditional statistics in modern science.
The author was transformed by reading "Behavior: The Control of Perception"(1973) and began a research program whose early years(?) seem to have been summarized in "Mind Readings: Experimental Studies of Purpose"(1992)
I don't understand what the quote is trying to say. What are the unrecognized consequences of the open-loop model?
It sounds like the author is upset that psychologists don't believe he has a model of behavior that explains 99% of some output variable using only one input variable. I'd have a hard time believing too.
This has been discussed here before.
The problem is that Marken's models don't actually have predictive power; he just fits a function to the data using as many free parameters as he has data points, and marvels at the perfect fit thus derived. One doesn't need to think highly of the current state of psychology to realize that Marken is a crank, and that any recognition Marken has in the PCT community is a sign that they are bereft of actual experimental support if not basic scientific reasoning skills.
The interaction you linked to was interesting. I didn't realize there was already a back story within this community with positions staked out and such. I offered the quote because it seemed like a beautifully mathematical objection to existing work that was "up this community's alley" but I haven't worked into the actual mathematics or experiments themselves. For example, I hadn't purchased either of the books that I linked to, not have I studied them - I simply assigned them high EV given the quality of the author's text.
Your comments, in the interaction you linked to, seem like a good arguments against Marken's theory (specifically the claim that his work involves more free parameters than data points appears to be a good argument against the theory, if true). However, in all of that back and forth, I noticed many links to "lesswrong heuristics" but I didn't notice any outside links to an actual research papers detailing methodology.
I'm substantially more ignorant on the subject than either you or your previous interlocutor and it took me a while to even understand that "PCT" was the theory Marken supports, that you two were taking the pro and con towards it, that your text was mostly between each other with a substantial amount of knowledge assumed. I wish you had both linked more, because it would have been educational.
That said, I'd like to see such links if you know of any. If I can swiftly dismiss Marken's work without further thought, that would be a very efficient use of time. Can you direct me to the links showing an example of his experimental work so I can verify that his research program is crippled by mathematical overfitting? The best I could find was Perceptual organization of behavior: A hierarchical control model of coordinated action but it was pay-walled so I can't access it now to look into it myself.
Through judicious abuse of my employer's resources, I have acquired a copy of the PDF - PM me an email address and I'll send it to you.
Thanks Robin! I have read this paper now, but it still doesn't seem to address the arguments that orthonormal linked to :-/
The 1986 study appeared to me to be basically well done, offering a fascinating paradigm that could be extended in many directions for further research with a reasonably strong result by itself. It basically confirmed the positive claims of Marken that hierarchical arrangements of negative feedback loop systems (designed, with a handful of optimized parameters, and then left alone) can roughly reproduce trained human behavior in a variety of dynamically changing toy domains, supporting the contention that whatever is operating in the human nervous system after a period of training is doing roughly the same effective computations as the model.
In the text, Merken addresses the "motor control literature" as making claims whose refutation was partly the purpose of his experiments.
It required a little more googling to figure out the claims he was trying to reject... but basically he seems to be objecting to the claim that mammals work as open loop controllers (that is, generating action signals based on an internal model of the world that are sent into the world with no verification loop or secondary corrections). This claim appears to have been founded mostly on things called "deafferentiation experiments"... which turned out to be aesthetically horrifying and also turned out to not actually prove the general case of "open loop" claims.
The most infamous of these experiments, (warning - kind of disturbing pictures) was basically:
The ability of monkeys mutilated in this fashion to (eventually?) move around purposively was taken as evidence that there was not a hierarchically arranged set of negative feedback motor control systems implemented in their nervous system. In practice (after the scientist was arrested for animal cruelty, PETA's request for custody was denied, and the monkeys were brainscanned, euthanized, and autopsied) it turned out that the monkey's brains had been massively re-wired by the experience. The practical upshot of the experiments seem to have primarily been to serve as dramatic evidence of adult primate brain plasticity (which they didn't believe in, back then?) rather than as confirmation of a negative feedback theory of motor control. (Probably there's more to it than that, but this is my first draft understanding.)
Merken dismisses these experiments in part by pointing out the difficulty of preventing negative feedback control processes if there are many sub controllers that can use measurements partially correlated to the measure being optimized and concludes with falsification examples and criteria for the general theory and the particular model that are not subject to this objection:
In short, I'm still impressed by Merken. His reasoning seems clean, his experiment, robust, his criticisms of motor-control and trait-theory, well reasoned. My very broad impression is that there may be a over-arching background argument here between "accurate model in the head producing aim and fire success" versus "incremental goal accomplishment via well tuned reflexes and continuous effort"? If that back story is operative then I guess my posterior probability was just pushed a little more in the direction of "reflexes and effort" over "models and plans".
If there is some trick still lurking here, Orthonormal, that you could point me to and spell out in detail rather than by reference to assertions and hand-waving rationality heuristics, that would be appreciated. The more time I spend on Merken's work, the more I find to appreciate. At this point, I've spend a day or two on this and I think the burden of proof is on you. If you take it up successfully I would be in your debt for rubbing a bit of sand out of my eyes :-)
The paper discussed in that interaction can be found here without a paywall.
As stated then (the conversation can be taken up from about here if not earlier), I think it's quite likely that simple control circuits can be found in facets of motor response; but Powers, Marken and Eby had been talking about control theory in cognitive domains (like akrasia) as if they could isolate simple circuits there, and my search for any kind of evidence turned up only this sort of embarrassing tripe.
And really, the math here is important— it's not a matter of disagreeing with interpretation, it's the plain fact that a generic model with 4 free parameters can be tweaked to precisely fit 4 data points, and it's clear from the paper that this is what Marken did. You simply need more data points than free parameters in order to generate any evidence in favor of a model; the fact that he never mentioned this, and instead crowed about the impressive fit of his model to the data, indicate either gross ignorance of how mathematical models work, or outright intent to mislead (coupled with an utterly incompetent peer review process.)
The gauntlet remains thrown, if anyone wants to point to an experimental study which demonstrates a discernible control circuit in a cognitive task (apart from tasks, like tracking a dot, which have an obvious motor component— in these, I do expect control circuits to be a good model for certain behavior). I would be surprised, but it would suffice to give credence to the theory in my eyes.
-- Marcel Proust, In Search of Lost Time
I'm a big fan of Ken Binmore, and this quote captures a lot of my dissatisfaction with LW's directions of inquiry. For example, it's more or less taken for granted here that future superintelligent AIs should cooperate on the Prisoner's Dilemma, so some of us set out to create a general theory of "superintelligent AIs" (including ones built by aliens, etc.) that would give us the answer we like.
Would it be correct to say you mean "should" in the wishful thinking sense of "we really want this outcome," rather than something normative or probabilistic?
Good question. The answer's yes, but now I'm wondering whether we really should expect alien-built AIs to be cooperators. I know Eliezer thinks we should.
Typo-hunt: should read "abandoning arithMetic" (without the capital of course)
Fixed.
"All things end badly - or else they wouldn't end"
Almost all relationships end in unhappiness or death. Or unhappiness leading to death.
A side note: All three of the quotes I've posted are from Binmore's Rational Decisions, which I'm about a third of the way through and have found very interesting. It makes a great companion to Less Wrong -- and it's also quite quotable in spots.
Dale Carnegie, How to Win Friends and Influence People
What if I am right 9 times out of 10 when I say I am 90% sure of something, but I am never or very rarely more than 50% sure of propositions of the form "This stock's price will go up/down, over a relevant time frame"?
Rephrase that and it sounds nonsensical: "If you can't outperform the stock market, then how can you be sure of anything?" I think Carnegie was just looking for a glib rationalization for his advice to avoid contradicting people whom you want to like you.
-- Ludwig von Mises, Epistemological Problems of Economics
This reminds me of B. F. Skinner's criticism of William James
Before he can add something of substance to the discussion of the epistemological problems of economics, Ludwig von Mises must look back in time, to previous events, and offer them as the explanation of why we want or desire things and why we also call those things agreeable or good.
I think Mises's point is rather that concepts like "good," "bad," "evil," "right," "wrong," "ought to" and "rights" all reduce back down to variations on "I desire it"/"It brings me pleasure" and the opposite. In other words, all ethical systems are dressed up (subjective) consequentialism and they only appear otherwise due to semantic confusion.
.
The response to that would be that you only do things that give others pleasure because the feeling of helping others is pleasurable to you or because you expect something in return, and that if neither of those were the case, you wouldn't do it. (I don't necessarily agree with that — I'm pretty sure I don't — but I do believe that's how they'd reduce it.)
Imagine that you got no satisfaction at all from bringing pleasure to others, but you did it anyway. What would be the reason?
Claude Lévi-Strauss
Alfred North Whitehead
Freeman Dyson
Bertrand Russell
The wizard who reads a thousand books is powerful. The wizard who memorizes a thousand books is insane.
"If A=B and B=C and C=D, then do not get a job proofreading." - Quid's Theorem
WIlliam Thomson, Lord Kelvin
Deleated as a repeat.
A repeat, but a good one.
Are the winners the only ones actually writing the history? We need to disabuse ourselves of this habit of saying things because they sound good. ----- Ta-Nehisi Coates
Coates runs a popular culture, black issues, and history blog with a very strong rationalist approach.
"In the animal kingdom, the rule is, eat or be eaten; in the human kingdom, define or be defined."
Thomas Szaz
--- Mark Liberman
http://friendlyatheist.com/2008/02/29/complete-the-atheist-joke-1/
My initial response was to chuckle, but when my analytical capacities kicked in a moment later I was disappointed.
If his initial assumptions was that he was walking into a bar, does that make him atheist in this metaphor? Substitute "walked into a bar" by "believed there is a god", the thing I assume it is a metaphor of. You will see it makes no sense.
I think it makes sense, as a poke at atheists.
Think about it this way. You walk into a bar, and you see no bartender. In your mind, you say "anything that is a bar will have a bartender. No bar tender, not a bar." Of course, the best thing to do before revising your assumptions is to wait for a bar tender. Maybe he/she is in the bathroom.
Similarly, if you claim there is no evidence of god that I've seen in my lifetime, you are using the wrong measure. Why should god (if there is one) make itself obvious during the short period that is a human lifetime.
This is almost an "irrationality quote" instead of a rationality quote, but still enlightening.
I was with you up until the "similarly". After that you start privileging the hypothesis - you should expect a god to make itself obvious during a human lifetime, by any description of a god ever proposed in history.
Many atheists were formerly theists.
Still, I suppose it might have been better as "A scientist walked into what he thought was a bar, but seeing no bartender, barstools, or drinks, he revised his initial assumption and decided he only walked into a room."
-- Clay Shirky
--via The Economist, "a saying of statisticians".
I like it, but do you have an issue number?
My father's been saying that as long as I can remember; he hasn't taken a statistics class since '82.
Never mind, then!
Here is the piece I got it from:
http://www.economist.com/specialreports/displaystory.cfm?story_id=15557465
"A different game: Information is transforming traditional businesses", Feb 25th 2010 - thanks!
-David Stevens
--Sir Edmund Hillary (1919-2008) - New Zealand Mountaineer and First man to Climb Mt. Everest
Interesting in light of pjeby's distinction between "you" and "yourself."
-Louis Aragon
"It is wrong always, everywhere, and for anyone, to believe anything upon insufficient evidence." ~William Kingdon Clifford
This is the quote that got me thinking about rationality as something other than "a word you use to describe things you believe so that you can deride those who disagree with you."
One of the most insidious sources of confusion, I find, is the distinction between the meaning of a word and its most frequent uses. It ties into the whole "Applause Lights" phenomenon, particularly "Fake Norms".
P.S. Belatedly: Welcome to Less Wrong! Feel free to introduce yourself in that thread.
"Face the facts. Then act on them. It's the only mantra I know, the only doctrine I have to offer you, and it's harder than you'd think, because I swear humans seem hardwired to do anything but. Face the facts. Don't pray, don't wish, don't buy into centuries-old dogma and dead rhetoric. Don't give in to your conditioning or your visions or your fucked-up sense of... whatever. FACE THE FACTS. THEN act."
--- Quellcrist Falconer, speech before the assault on Millsport. (Richard Morgan, Broken Angels)
Seen on bumper sticker, via ^zhurnaly.
This is more important than it looks. Most people's beliefs are just recorded memes that bubbled up from their subconscious when someone pressed them for their beliefs. They wonder what they believe, their mind regurgitates some chatter they heard somewhere, and they go, "Aha, that must be what I believe." Unless they take special countermeasures, humans are extremely suggestible.
-- Wayne Gretzky (but I've seen it attributed to Michael Jordan and Joe Ledbetter, HS coach)