= 27d567bc2d48d9f40e9ccc20ea370b15)
Rationality Quotes September 2013
Another month has passed and here is a new rationality quotes thread. The usual rules are:
- Please post all quotes separately, so that they can be upvoted or downvoted separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)
- Do not quote yourself.
- Do not quote from Less Wrong itself, HPMoR, Eliezer Yudkowsky, or Robin Hanson. If you'd like to revive an old quote from one of those sources, please do so here.
- No more than 5 quotes per person per monthly thread, please.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
Comments (456)
-- Albert Einstein
These (nebulous) assertions seem unlikely on many levels. Psychopaths have few morals but continue to exist. I have no idea what "inner balance" even is.
He may be asserting that morals are necessary for the existence of humanity as a whole, in which case I'd point to many animals with few morals who continue to exist just fine.
I know of no animals other than humans who have nuclear weapons and the capacity to completely wipe themselves out on a whim.
The existence of nuclear weapons should be taken as evidence that humans are not very moral. (And yet survive so far.)
Consider what "the cold war" might have been like if we hadn't of had nuclear weapons. It probably would have been less cold. Come to think of it, cold wars are the best kind of wars. We could use more of them.
Yes nukes have done terrible things, could have done far worse, and still might. However since their invention conventional weapons have still killed far, far more people. We've seen plenty of chances for countries to use nukes where they've not, so I think its safe to say the existence of nukes isn't on average more dangerous than the existence of other weapons. The danger in them seems to come from the existential risk which is not present when using conventional weapons.
Indeed, I'm pretty sure that if not for nuclear weapons, some right-thinking Russian would have declared war over the phrase "hadn't of had". And very rightly so. The slaughter inflicted by mere armies of millions, with a few tens of thousands of tanks, would have been a small price to pay to rid the world of abominations like that one.
Huh? Can you unpack this for me, I don't see how it can make sense.
Einstein is not saying that humans are necessarily moral, but rather that they ought to be moral.
Furthermore, it is arguable that nuclear weapons are not necessarily immoral in and of themselves. Like any tool or weapon, they can be used for moral and immoral ends. For instance, nuclear weapons may well be one of the most effective means of destroying Earth-directed masses such as Existential Risk threatening asteroids. They may also be extremely effective for deterring conventional warfare between major powers.
The only previous actual use of nuclear weapons against human targets was for the ends of ending a world war, and it did so rather successfully. That we have chosen not to use nuclear weapons irresponsibly may well suggest that those with the power to wield nuclear weapons have in fact been more morally responsible than we give them credit.
True, but its not clear morals have saved us from this. Many of our morals emphasize loyalty to our own groups (e.g. the USA) over our out groups (e.g. the USSR), with less than ideal results. I think if I replaced "morality" with "benevolence" I'd find the quote more correct. I likely read it too literally.
Though the rest of it still doesn't make any sense to me.
Checking Google failed to yield an original source cited for this quote.
-- John Scalzi
So is the failure mode of many people who are not, and don't hold themselves to be, clever. I fail to see the correlation.
ETA: Scalzi addresses a very specific topic, and even then he really seems to address some specific anecdote that he doesn't share. I don't think it's a rationality quote.
--Mr. Gradgrind, from Hard Times by Charles Dickens.
The character is portrayed as a villain, but this quote struck me as fair (if you take a less confused view of "Facts" than Gradgrind).
http://xkcd.com/863/
It looks to me like you're making the sophisticated point that some facts vary in usefulness. I agree.
The point being made by Gradgrind is much more basic: children should focus on Fact over Fancy. As an example, he refuses to teach his children fairy tales, deciding that they should learn science instead. (Unfortunately, Dickens presents science as dull collections in cabinets, and so the children are rather put out by this.)
-- Norman Page, Auden and Isherwood: The Berlin Years
Most don't even know why they believe what they believe, man
Never taking a second to look at life
Bad water in our seeds, y'all, still growing weeds, dawg
-- CunninLynguists featuring Immortal Technique, Never Know Why, A Piece of Strange (2006)
Slightly edited from Scott Adams' blog.
And a similar sentiment from SMBC comics.
I personally can't see how a monkey turns into a human. But that's irrelevant because that is not the claim of natural selection. This makes a strawman of most positions that endorse something approximately like free will. Also:
Just the legal system? Gah. Everybody on earth does this about 200 times a day.
-- Gordon R. Dickson, "The Tactics of Mistake".
Q: Why are Unitarians lousy singers? A: They keep reading ahead in the hymnal to see if they agree with it.
Bizarro Blog
The reason is that banning certain words works much better as a Schelling point.
I don't buy it. Even if we accept for the sake of argument that limiting sexual references on broadcast TV is a good plan (a point that I don't consider settled, by the way), using dirty words as a proxy runs straight into Goodhart's law: the broadcast rules are known in advance, and innuendo's bread and butter to TV writers. A good Schelling point has to be hard to work around, even if you can't draw a strict line; this doesn't qualify.
Better for what, and better than what alternatives?
You wind up in endless arguments about whether this particular show is beyond the pail.
That doesn't seem like it answers my question.
What's the goal in this case? This sounds like it's only attempting to address effectiveness at avoiding disputes over standards, but that could more easily be achieved by not having any restrictions at all.
Sorry, this is nonsense. It's not hard to Google up a copy of the FCC rules. http://www.fcc.gov/guides/obscenity-indecency-and-profanity :
I am fairly sure that "I’m going to rape your 8-year-old daughter with a trained monkey" would count as describing sexual activities in patently offensive terms, and would not be allowed when direct use of swear words would not be allowed. Just because you don't use a list of words doesn't mean that what you say will be automatically allowed.
Furthermore, the Wikipedia page on the seven words ( http://en.wikipedia.org/wiki/Seven_dirty_words ) points out that " The FCC has never maintained a specific list of words prohibited from the airwaves during the time period from 6 a.m. to 10 p.m., but it has alleged that its own internal guidelines are sufficient to determine what it considers obscene." It points out cases where the words were used in context and permitted.
In other words, this quote is based on a sound-bite distortion of actual FCC behavior and as inaccurate research, is automatically ineligible to be a good rationality quote.
What is the basis for you being sure?
Howard Stern, a well-known "shock jock" spent many years on airwaves regulated by the FCC. He more or less specialized in "describing sexual activities in patently offensive terms" and while he had periodic run-ins with the FCC, he, again, spent many years doing this.
The FCC rule is deliberately written in a vague manner to give the FCC discretionary power. As a practical matter, the seven dirty words are effectively prohibited by FCC and other offensive expressions may or may not be prohibited. Broadcasters occasionally test the boundaries and either get away with it or get slapped down.
Yes, and this illustrates another problem: if we agreed on what to ban, it would make more sense to use discretionary human judgment than rules which might be manipulated or Munchkin-ed. We don't agree.
I do think it would make sense in the abstract to ban speech if we had scientific reason to think it harmed people, the way we had reason to think leaded gasoline harmed people in the 1920s. But I only know one class of speech where that might apply, and it'll never get on TV anyway. ^_^
For fairness sake.
But it's not who you are underneath, it's what you do that defines you.
-Rachel Dawes, Batman Begins
-Hermann Hesse, The Glass Bead Game
Saturday Morning Breakfast Cereal
Roger Ebert
Would be nice if this were true.
It's probably true for academic film theory. I mean how hard could it really be?
Peter Shor replying in the comment section of Scott Aaronson's blog post Firewalls.
Friedrich Nietzsche
Because it's really really useful?
Jeez, people really don't appreciate poetic language around here, huh?
(That would probably be close to my answer too, I'm just a little stunned by all the downvotes.)
Dan Ariely, Predictably Irrational: The Hidden Forces that Shape Our Decisions, New York, 2008, pp. 171-172
In my experience, who started the conflict, who is to blame, etc. is explicitly taught as fact to each side's children. Israelis and Palestinians don't agree on facts at all. A civilized discussion of politics generally requires agreeing not to discuss most past facts.
Paul Graham
Y'know, there are all sorts of counterexamples to this ... but I think its still a bad sign, if not a definitive one, on the basis that if I had been more suspicious of things I was talking myself into I would have had a definite net benefit to my life. (Not counting times I was neurohacking myself, admittedly, but that's not really the same.)
Yes, there's an unfortunate tendency among some "rationalist" types to dismiss heuristics because they don't apply in every situation.
Nick Szabo
Is this a similar message to Penn Jillette saying:
"If you don’t pay your taxes and you don’t answer the warrant and you don’t go to court, eventually someone will pull a gun. Eventually someone with a gun will show up. "
or did I miss the boat?
Well, it's similar, but for two differences:
1) It uses a different and wider category of examples. Viz. "initiate force [...] to compel them to hand over goods, to let us search their property, or to testify."
2) It makes a consequentialist claim about forcing people to e.g. let us search their property for evidence: "we can't properly respond to a global initiation of force without local initiations of force."
The second difference here is important because it directly contradicts the typical libertarian claim of "if we force people to do things much less than we currently do, that will lead to good consequences." The first difference is rhetorically important because it is a place where people's gut reaction is more likely to endorse the use of force, and people have been less exposed to memes about forcibly searching peoples' property (compared to the ubiquity of people disliking taxes) that would cause them to automatically respond rather than thinking.
Actually that isn't what Szabo is saying. His point is to contradict the claim of the anarcho-capitalists that "if we never force people to do things, that will lead to good consequences."
Breaking Bad, episode Rabid Dog.
(Although "won't" should be "can't".)
Depending on how the violence is applied, it can also make it better.
"[G]et wisdom: and with all thy getting, get understanding." -- Proverbs 4:7
Plato
In a democratic republic of over 300 million people, whether or not you "participate in politics" has virtually no effect on whether your rulers are inferior or superior than yourself (unless "participate in politics" is a euphemism for coup d'état).
A democratic republic is not necessary. In any kind of political regime encompassing 300 million people, your participation in politics has very small expected effect on whether your rulers are inferior to you.
Another case of rationalists failing at collective action.
It's not a nation of 300 million rationalists, however.
Yet.
And you don't even need a majority of rationalists by headcount. You just need to find and hack the vulnerable parts of your culture and politics where you have a chance of raising people's expectations for rational decision making. Actual widespread ability in rationality skills comes later.
Whenever you feel pessimistic about moving the mean of the sanity distribution, try reading the Bible or the Iliad and see how far we've come already.
People don't expect rational decision making from politics, because that's not what politics is for. Politics exists for the sake of power (politics), coordination and control, and of tribalism, not for any sort of decision making. When politicians make decisions, they optimize for political purposes, not for anything external such as economic, scientific, cultural, etc. outcomes.
When people try make decisions to optimize something external like that, we don't call them politicians; we call them bureaucrats.
If you tried to do what you suggest, you would end up trying not to improve or reform politics, but to destroy destroy it. Good luck with that.
Depends on who "we" are. A great many people still believe in the Bible and try to emulate it, or other comparable texts.
A little cynical maybe? Politicians don't spend 100% of the time making decisions for purely political reasons. Sometimes they are trying to achieve something, even if broadly speaking the purposes of politics are as you imply.
But of course, most of the people we would prefer to be more rational don't know that's what politics is for, so they aren't hampered by that particular excuse to give up on it. Anyway, they could quite reasonably expect more rational decision making from co-workers, doctors, teachers and others.
I don't think the people making decisions to optimise an outcome are well exemplified by bureaucrats. Try engineers.
Knowing that politics is part of what people do, and that destroying it is impossible, yes I would be trying to improve it, and hope for a more-rational population of participants to reform it. I would treat a claim that the way it is now is eternal and unchangeable as an extraordinary one that's never been true so far. So, good luck with that :)
You aren't seriously suggesting the mean of the sanity distribution hasn't moved a huge amount since the Bible was written? Or even in the last 100 years? I know I'm referring to a "sanity distribution" in an unquantifiable hand-wavy way, but do you doubt that those people who believe in a literalist interpretation of the Bible are now outliers, rather that the huge majority they used to be?
Jon Stewart, talking to Richard Dawkins (S18, E156)
Let's get one thing straight: ignorance killed the cat.
Curiosity was framed.
Theophanis the Monk, "The Ladder of Divine Grace"
--- Sir Hubert Parry, speaking to The Royal College of Music about the purpose of music examinations
Initially I thought this a wonderful quote because, looking back at my life, I could see several defeats (not all in music) attributable to sipping and sampling. But Sir Hubert is speaking specifically about music. The context tells you Sir Hubert's proposed counter to sipping and sampling: individual tuition aiming towards an examination in the form a viva.
The general message is "counter the tendency to sipping and sampling by finding something definite to work for, analogous to working ones way up the Royal College of Music grade system". But working out the analogy is left as an exercise for the reader, so the general message, if Sir Hubert intended it at all, is rather feeble.
That's an surprisingly forgiving thing to say. She lives in a place where eating legs to prevent starvation is a venerable military tradition, and a non-zero number of people end up in the Girls' Working School.
However, to set yourself against all the stupidity in the world is an insurmountable task.
-- Richard James, founding priest of a Toronto based Wicca church, quoted in a thegridto article
by Hannes Leitgeb, from his joint teaching course with Stephan Hartmann (author of Bayesian Epistemology) on Coursera entitled 'An Introduction to Mathematical Philoosphy'.
The course topics are "Infinity, Truth, Rational Belief, If-Then, Confirmation, Decision, Voting, and Quantum Logic and Probability". In many ways, a very LW-friendly course, with many mentions and discussions of people like Tarski, Gödel etc.
-- TychoCelchuuu on Reddit
That's not even an example of the ad hominem fallacy.
"You have an ugly face, so you're wrong" is ad hominem. "You have an ugly face" is not. It's just a statement. Did the speaker imply the second part? Maybe... but probably not. It was probably just an insulting rejoinder.
Insults, i.e. "Attacking you, not your argument", is not what ad hominem is. It's a fallacy, remember? It's no error in reasoning to call a person ugly. Only when you conclude from this that they are wrong do you commit the fallacy.
So:
A: It's wrong to stab your neighbor and take their stuff.
B: Your face is ugly.
A: The ugliness of my face has no bearing on moral...
B, interrupting: Didn't say it does! Your face is still ugly!
Fallacy names are useful for the same reason any term or technical vocab are useful.
'But notice how you could've just you meant the quantity 1+1+1+1 without yelling "four" first! In fact, that's how all 'numbers' work. If someone is actually using a quantity, you can just give that quantity directly without being a mathematician and finding a pat little name for all of their quantities used.'
I voted your comment up because I agree that the vocabulary is useful for both the person committing the fallacy and (I think this is overlooked) for the person recognizing the fallacy.
However, I think the point of the original quote is probably that when someone points out a fallacy they are probably felling angry and want to insult their interlocutor.
Fallacy names are great for chunking something already understood. The problem is that most people who appeal to them don't understand them, and therefore mis-use them. If they spoke in descriptive phrases rather than in jargon, there would be less of an illusion of transparency and people would be more likely to notice that there are discrepancies in usage.
For instance, most people don't understand that not all personal attacks are ad hominem fallacies. The quotation encourages that particular mistake, inadvertently. So it indirectly provides evidence for its own thesis.
Another good one from the same source:
Paul Graham
Richard Mitchell - Less Than Words Can Say
idontknowbut@gmail.com
The same is broadly true of e.g. pop music or politics: you can't really escape them. It's not necessarily a reason to study them, though.
This seems true. What I am curious about is whether it remains true if you substitute "don't" with "do". Those that do study philosophy have not on average impressed me with their ability to discriminate among the bullshit.
Anonymous, found written in the Temple at 2013 Burning Man
Part of that seems to be from HPMOR. I'm not sure where the rest comes from.
Yeah, almost certainly HPMOR inspired. Eliezer's work has spread far.
Rationality wakes up last:
Scott Adams on waking up with a numb arm.
Never experienced this exact experience - I don't sleep on my arm - but waking up stupid? Definitely.
I woke up one time with both arms completely numb. I tried to turn the light on and instead fell out of bed. I felt certain that I was going to die right then.
Odd, this has never happened to me. Not the experience of waking up with a numb arm, but the experience of being at all worried about it.
I was quite worried the first time I experienced a numb arm which was both completely dead to sensation and totally immobile for multiple minutes, but after that had happened before, successive occurrences were no longer particularly worrying.
I've experienced 'pins and needles' many times, but a totally 'dead' arm only once. I didn't have any control over it, and when I tried to move it I hit myself in the nose. Quite hard, too!
When I experienced a "totally dead arm," I didn't just not have control over it, I couldn't even wiggle my fingers. It was pretty frightening, since as far as I knew the arm might have experienced extensive cell death from blood deprivation; after all, I had no sign of it being operational at all. My circulation was poor enough that I couldn't even tell if it was still warm, beyond residual heat from my lying on it.
It's happened twice again since then though, and the successive occasions were not particularly distressing.
IIRC the numbness is caused by nerve compression, not blood-flow cutoff.
edit: Apparently it can be either way: http://www.wisegeek.org/what-are-the-most-common-causes-of-numbness-while-sleeping.htm
edit2: And another source claims it's due to nerves, so I dunno. I do find the nerve explanation more plausible than the blood-flow one.
I couldn't do anything with the arm either, it just felt as if it wasn't there. It was decades ago, but I think I used my shoulder muscles to try and move it. I was probably scared too, but that part of the memory is quite vague.
Scott Adams
G K Chesterton
I don't think that's the case. There are plenty of shy intellectuals who don't push their ideas on other people. Darwin sat more than a decade on his big idea.
There are ideas that are about qualia. It doesn't make much sense to try to explain a blind person what red looks like and the same goes for other ideas that rest of observed qualia instead of resting on theory. If I believe in a certain idea because I experienced a certain qualia and I have no way of giving you the experience of the same qualia, I can't explain you the idea. In some instances I might still try to explain the blind what red looks like but there are also instance where I see it as futile.
One way of teaching certain lessons in buddhism is to give a student a koan that illustrates the lesson and let him meditate over the koan for hours. I don't see anything dishonest about teaching certain ideas that way.
If someone thinks about a topic in terms of black and white it just takes time to teach him to see various shades of grey.
Thomas Edison
— Jack Vance, The Languages of Pao
--"Adventure Time" episode "The Businessmen": the zombie businessmen are explaining why they are imprisoning soft furry creatures in a glass bowl.
"To know thoroughly what has caused a man to say something is to understand the significance of what he has said in its very deepest sense." -Willard F. Day
Ted Chiang, Tower of Babylon
Black Books, Elephants and Hens. H/t /u/mrjack2 on /r/hpmor.
-Leonard Susskind, Susskind's Rule of Thumb
Hm. A generalized phenomenon of overwhelming physicist underconfidence could account for a reasonable amount of the QM affair.
Great quote.
Unfortunately, we find ourselves in a world where the world's policy-makers don't just profess that AGI safety isn't a pressing issue, they also aren't taking any action on AGI safety. Even generally sharp people like Bryan Caplan give disappointingly lame reasons for not caring. :(
After reading Robin's exposition of Bryan's thesis, I would disagree that his reasons are disappointingly lame.
Which could either indicate that the reasons are good or that your standards are lower than Luke's and so trigger no disappointment.
Bryan is expressing a "standard economic intuition" but... did you see Carl's comment reply on Caplan's post, and also mine?
I did see Eelco Hoogendoorn 's and it is absolutely spot on.
I'm hardly a fan of Caplan, but he has some Bayesianism right:
Based on how things like this asymptote or fail altogether, he has a low prior for foom.
He has low expectation of being able to identify in advance (without the work equivalent to the creation of the AI) exact mechanisms by which it is going to asymptote or fail, irrespective of whenever it does or does not asymptote or fail, so not knowing such mechanisms does not bother him a whole lot.
Even assuming he is correct he expects a plenty of possible arguments against this position (which are reliant on speculations), as well as expects to see some arguers, because the space of speculative arguments is very huge. So such arguments are not going to move him anywhere.
People don't do that explicitly any more than someone who's playing football is doing Newtonian mechanics explicitly. Bayes theorem is no less fundamental than the laws of motion of the football.
Likewise for things like non-testability: nobody's doing anything explicitly, it is just the case that due to something you guys call "conservation of expected evidence" , when there is no possibility of evidence against a proposition, then a possibility of evidence in favour of the proposition would violate the Bayes theorem.
I'm not sure how you could have such a situation, given that absence of expected evidence is evidence of the absence. Do you have an example?
Well, the probabilities wouldn't be literally zero. What I mean is that lack of a possibility of strong evidence against something, and only a possibility of very weak evidence against it (via absence of evidence) implies that strong evidence in favour of it must be highly unlikely. Worse, such evidence just gets lost among the more probable 'evidence that looks strong but is not'.
Ah, I think I follow you.
Absence of evidence isn't necessarily a weak kind of evidence.
If I tell you there's a dragon sitting on my head, and you don't see a dragon sitting on my head, then you can be fairly sure there's not a dragon on my head.
On the other hand, if I tell you I've buried a coin somewhere in my magical 1cm deep garden - and you dig a random hole and don't find it - not finding the coin isn't strong evidence that I've not buried one. However, there there's so much potential weak evidence against. If you've dug up all but a 1cm square of my garden - the coin's either in that 1cm or I'm telling porkies, and what are the odds that - digging randomly - you wouldn't have come across it by then? You can be fairly sure, even before digging up that square, that I'm fibbing.
Was what you meant analogous to one of those scenarios?
Yes, like the latter scenario. Note that the expected utility of digging is low when the evidence against from one dig is low.
edit: Also. In the former case, not seeing a dragon sitting on your head is very strong evidence against there being a dragon. Unless you invoke un-testable invisible dragons which may be transparent to x-rays, let dust pass through it unaffected, and so on. In which case, I should have a very low likelihood of being convinced that there is a dragon on your head, if I know that the evidence against would be very weak.
edit2: Russel's teapot in the Kuiper belt is a better example still. When there can be only very weak evidence against it, the probability of encountering or discovering strong evidence in favour of it must be low also, making it not worth while to try to come up with evidence that there is a teapot in the Kuiper belt (due to low probability of success), even when the prior probability for the teapot is not very low.
Then, to extend the analogy: Imagine that digging has potentially negative utility as well as positive. I claim to have buried both a large number of nukes and a magical wand in the garden.
In order to motivate you to dig, you probably want some evidence of magical wands. In this context that would probably be recursively improving systems where, occasionally, local variations rapidly acquire super-dominance over their contemporaries when they reach some critical value. Evolution probably qualifies there - other bipedal frames with fingers aren't particularly dominant over other creatures in the same way that we are, but at some point we got smart enough to make weapons (note that I'm not saying that was what intelligence was for though) and from then on, by comparison to all other macroscopic land-dwelling forms of life, we may as well have been god.
And since then that initial edge in dominance has only ever allowed us to become more dominant. Creatures afraid of wild animals are not able to create societies with guns and nuclear weapons - you'd never have the stability for long enough.
In order to motivate you not to dig, you probably want some evidence of nukes. In this context, recursively - I'm not sure improving is the right word here - systems with a feedback state, that create large amounts of negative value. Well, to a certain extent that's a matter of perspective - from the perspective of extinct species the ascendancy of humanity would probably not be anything to cheer about, if they were in a position to appreciate it. But I suspect it can at least stand on its own that it tends to be the case that failure cascades are easier to make than cascade successes. One little thing goes wrong on your rocket and then the situation multiplies; a small error in alignment rapidly becomes a bigger one; or the timer on your patriot battery is losing a fraction of a second and over time your perception of where the missiles are is off significantly. - it's only with significant effort that we create systems where errors don't multiply.
(This is analogous to altering your expected value of information - like if earlier you'd said you didn't want to dig and I'd said, 'well there's a million bucks there' instead - you'd probably want some evidence that I had a million bucks, but given such evidence the information you'd gain from digging would be worth more.)
This seems to be fairly closely analogous to Elizer's claims about AI, at least if I've understood them correctly, that we have to hit an extremely small target and it's more likely that we're going to blow ourselves to itty-bitty pieces/cover the universe in paperclips if we're just fooling around hoping to hit on it by chance.
If you believe that such is the case, then the only people you're going to want looking for that magic wand - if you let anyone do it at all - are specialists with particle detectors - indeed if your garden is in the middle of a city you'll probably make it illegal for kids to play around anywhere near the potential bomb site.
Now, we may argue over quite how strongly we have to believe in the possible existence of magitech nukes to justify the cost of fencing off the garden - personally I think the statement:
Is to constrain what you'll accept for potential evidence pretty dramatically - we're talking about systems in general, not just individual people, and recursively improving systems with high asymptotes relative to their contemporaries have happened before.
It's not clear to me that the second claim he makes is even particularly meaningful:
Sure, I think that they probably won't go to infinity - but I don't see any reason to suspect that they won't converge on a much higher value than our own native ability. Pretty much all of our systems do, from calculators to cars.
We can even argue over how you separate the claims that something's going to foom from the false claims of such (I'd suggest, initially, just seeing how many claims that something was going to foom have actually been made within the domain of technological artefacts, it may be that the base-line credibility is higher than we think.) But that's a body of research that Caplan, as far as I'm aware, hasn't forwarded. It's not clear to me that it's a body of research with the same order of difficulty as creating an actual AI either. And, in its absence, it's not clear to me that to answer in effect, "I'll believe it when I see the mushroom cloud." is a particularly rational response.
Why won't you update towards the possibility that they're right and you're wrong?
This model should rise up much sooner than some very low prior complex model where you're a better truth finder about this topic but not any topic where truth-finding can be tested reliably*, and they're better truth finders about topics where truth finding can be tested (which is what happens when they do their work), but not this particular topic.
(*because if you expect that, then you should end up actually trying to do at least something that can be checked because it's the only indicator that you might possibly be right about the matters that can't be checked in any way)
Why are the updates always in one direction only? When they disagree, the reasons are "lame" according to yourself, which makes you more sure everyone's wrong. When they agree, they agree and that makes you more sure you are right.
It's not so much that I'm a better truth finder, it's that I've had the privilege of thinking through the issues as a core component of my full time job for the past two years, and people like Caplan only raise points that have been accounted for in my model for a long time. Also, I think the most productive way to resolve these debates is not to argue the meta-level issues about social epistemology, but to have the object-level debates about the facts at issue. So if Caplan replies to Carl's comment and my own, then we can continue the object-level debate, otherwise... the ball's in his court.
This doesn't appear to be accurate. E.g. Carl & Paul changed my mind about the probability of hard takeoff. And when have I said that some public figure agreeing with me made me more sure I'm right? See also my comments here.
If I mention a public figure agreeing with me, it's generally not because this plays a significant role in my own estimates, it's because other people think there's a stronger correlation between social status and correctness than I do.
Yes, but why Caplan did not see it fit to think about the issue for a significant time, and you did?
There's also the AI researchers who have had the privilege of thinking about relevant subjects for a very long time, education, and accomplishments which verify that their thinking adds up over time - and who are largely the actual source for the opinions held by the policy makers.
By the way, note that the usual method of rejection of wrong ideas, is not even coming up with wrong ideas in the first place, and general non-engagement of wrong ideas. This is because the space of wrong ideas is much larger than the space of correct ideas.
What I expect to see in the counter-factual world where the AI risk is a big problem, is that the proponents of the AI risk in that hypothetical world have far more impressive and far more relevant accomplishments and credentials.
The first problem with highly speculative topics is that great many arguments exist in favour of either opinion on a speculative topic. The second problem is that each such argument relies on a huge number of implicit or explicit assumptions that are likely to be violated due to their origin as random guesses. The third problem is that there is no expectation that the available arguments would be a representative sample of the arguments in general.
Hmm, I was under the impression that you weren't a big supporter of the hard takeoff to begin with.
Well, your confidence should be increased by the agreement; there's nothing wrong with that. The problem is when it is not balanced by the expected decrease by disagreement.
There are a great many differences in our world model, and I can't talk through them all with you.
Maybe we could just make some predictions? E.g. do you expect Stephen Hawking to hook up with FHI/CSER, or not? I think... oops, we can't use that one: he just did. (Note that this has negligible impact on my own estimates, despite him being perhaps the most famous and prestigious scientist in the world.)
Okay, well... If somebody takes a decent survey of mainstream AI people (not AGI people) about AGI timelines, do you expect the median estimate to be earlier or later than 2100? (Just kidding; I have inside information about some forthcoming surveys of this type... the median is significantly sooner than 2100.)
Okay, so... do you expect more or fewer prestigious scientists to take AI risk seriously 10 years from now? Do you expect Scott Aaronson and Peter Norvig, within 25 years, to change their minds about AI timelines, and concede that AI is fairly likely within 100 years (from now) rather than thinking that it's probably centuries or millennia away? Or maybe you can think of other predictions to make. Though coming up with crisp predictions is time-consuming.
Well, I too expect some form of something that we would call "AI", before 2100. I can even buy into some form of accelerating progress, albeit the progress would be accelerating before the "AI" due to the tools using relevant technologies, and would not have that sharp of a break. I even do agree that there is a certain level of risk involved in all the future progress including progress of the software.
I have a sense you misunderstood me. I picture this parallel world where legitimate, rational inferences about the AI risk exist, and where this risk is worth working at in 2013 and stands out among the other risks, as well as any other pre-requisites for making MIRI worthwhile hold. And in this imaginary world, I expect massively larger support than "Steven Hawkins hooked up with FHI" or what ever you are outlining here.
You do frequently lament that the AI risk is underfunded, under-supported, and there's under-awareness about it. In the hypothetical world, this is not the case and you can only lament that the rational spending should be 2 billions rather than 1 billion.
edit: and of course, my true rejection is that I do not actually see rational inferences leading there. The imaginary world stuff is just a side-note to explain how non-experts generally look at it.
edit2: and I have nothing against FHI's existence and their work. I don't think they are very useful, or address any actual safety issues which may arise, though, but with them I am fairly certain they aren't doing any harm either (Or at least, the possible harm would be very small). Promoting the idea that AI is possible within 100 years, however, is something that increases funding for AI all across the board.
Right, this just goes back to the same disagreement in our models I was trying to address earlier by making predictions. Let me try something else, then. Here are some relevant parts of my model:
The question should not be about "highly credentialed" people alone, but about how they fare compared to people who are rather very low "credentialed".
In particular, on your list, I expect people with fairly low credentials to fare much worse, especially at identification of the important issues as well as on rational thinking. Those combine multiplicatively, making it exceedingly unlikely - despite the greater numbers of the credential-less masses - that people who lead the work on an important issue would have low credentials.
What's EA? Effective altruism? If it's an existential risk, it kills everyone, selfishness suffices just fine.
Ohh, come on. That is in no way a demonstration that insurance companies in general follow faulty strategies, and especially is not a demonstration that you could do better.
Indeed.
No doubt! I wasn't comparing highly credentialed people to low-credentialed people in general. I was comparing highly credentialed people to Bostrom, Yudkowsky, Shulman, etc.
A selfish person protecting against existential risk builds a bunker and stocks it with sixty years of foodstuffs. That doesn't exactly help much.
Luke, why are you arguing with Dmytry?
This is why many scientists are terrible philosophers of science. Not all of them, of course; Einstein was one remarkable exception. But it seems like many scientists have views of science (e.g. astonishingly naive versions of Popperianism) which completely fail to fit their own practice.
Yes. When chatting with scientists I have to intentionally remind myself that my prior should be on them being Popperian rather than Bayesian. When I forget to do this, I am momentarily surprised when I first hear them say something straightforwardly anti-Bayesian.
Examples?
Statements like "I reject the intelligence explosion hypothesis because it's not falsifiable."
I see. I doubt that it is as simple as naive Popperianism, however. Scientists routinely construct and screen hypotheses based on multiple factors, and they are quite good at it, compared to the general population. However, as you pointed out, many do not use or even have the language to express their rejection in a Bayesian way, as "I have estimated the probability of this hypothesis being true, and it is too low to care." I suspect that they instinctively map intelligence explosion into the Pascal mugging reference class, together with perpetual motion, cold fusion and religion, but verbalize it in the standard Popperian language instead. After all, that is how they would explain why they don't pay attention to (someone else's) religion: there is no way to falsify it. I suspect that any further discussion tends to reveal a more sensible approach.
Yeah. The problem is that most scientists seem to still be taught from textbooks that use a Popperian paradigm, or at least Popperian language, and they aren't necessarily taught probability theory very thoroughly, they're used to publishing papers that use p-value science even though they kinda know it's wrong, etc.
So maybe if we had an extended discussion about philosophy of science, they'd retract their Popperian statements and reformulate them to say something kinda related but less wrong. Maybe they're just sloppy with their philosophy of science when talking about subjects they don't put much credence in.
This does make it difficult to measure the degree to which, as Eliezer puts it, "the world is mad." Maybe the world looks mad when you take scientists' dinner party statements at face value, but looks less mad when you watch them try to solve problems they care about. On the other hand, even when looking at work they seem to care about, it often doesn't look like scientists know the basics of philosophy of science. Then again, maybe it's just an incentives problem. E.g. maybe the scientist's field basically requires you to publish with p-values, even if the scientists themselves are secretly Bayesians.
If there was a genuine philosophy of science illumination it would be clear that, despite the shortcomings of the logical empiricist setting in which Popper found himself , there is much more of value in a sophisticated Popperian methodological falsificationism than in Bayesianism. If scientists were interested in the most probable hypotheses, they would stay as close to the data as possible. But in fact they want interesting, informative, risky theories and genuine explanations. This goes against the Bayesian probabilist ideal. Moreover, you cannot falsify with Bayes theorem, so you'd have to start out with an exhaustive set of hypotheses that could account for data (already silly), and then you'd never get rid of them---they could only be probabilistically disconfirmed.
Strictly speaking, one can't falsify with any method outside of deductive logic -- even your own Severity Principle only claims to warrant hypotheses, not falsify their negations. Bayesian statistical analysis is just the same in this regard.
A Bayesian analysis doesn't need to start with an exhaustive set of hypotheses to justify discarding some of them. Suppose we have a set of mutually exclusive but not exhaustive hypotheses. The posterior probability of an hypothesis under the assumption that the set is exhaustive is an upper bound for its posterior probability in an analysis with an expanded set of hypotheses. A more complete set can only make a hypotheses less likely, so if its posterior probability is already so low that it would have a negligible effect on subsequent calculations, it can safely be discarded.
I'm a Bayesian probabilist, and it doesn't go against my ideal. I think you're attacking philosophical subjective Bayesianism, but I don't think that's the kind of Bayesianism to which lukeprog is referring.
For what it's worth, I understand well the arguments in favor of Bayes, yet I don't think that scientific results should be published in a Bayesian manner. This is not to say that I don't think that frequentist statistics is frequently and grossly mis-used by many scientists, but I don't think Bayes is the solution to this. In fact, many of the problems with how statistics is used, such as implicitly performing many multiple comparisons without controlling for this, would be just as large of problems with Bayesian statistics.
Either the evidence is strong enough to overwhelm any reasonable prior, in which case frequentist statistics wlil detect the result just fine; or else the evidence is not so strong, in which case you are reduced to arguing about priors, which seems bad if the goal is to create a societal construct that reliable uncovers useful new truths.
No, the multiple comparisons problem, like optional stopping, and other selection effects that alter error probabilities are a much greater problem in Bayesian statistics because they regard error probabilities and the sampling distributions on which they are based as irrelevant to inference, once the data are in hand. That is a consequence of the likelihood principle (which follows from inference by Bayes theorem). I find it interesting that this blog takes a great interest in human biases, but guess what methodology is relied upon to provide evidence of those biases? Frequentist methods.
Deborah, what do you think of jsteinhardt's Beyond Bayesians and Frequentists?
But why not share likelihood ratios instead of posteriors, and then choose whether or not you also want to argue very much (in your scientific paper) about the priors?
Oglaf (Original comic NSFW)
How have I been reading Oglaf for so long without knowing about the epilogues?
... the what.
Ahh I just finished that.
For anyone unaware, SMBC has an additional joke panel when you mouse over the red button at the bottom
Actually, you have to click it now. Just a heads up to anyone reading this and trying to find them.
...oh crap, I'm going to have to reread the whole thing, aren't I.
And the mouseovers. And the alt text, which is different again.
And the mock ads at the bottom.
ETA: Explanation: Sometimes the banner at the bottom will contain an actual (randomized) ad, but many of the comics have their own funny mock ad associated. (When I noticed this, I went through all the ones I had already read again, to not miss out on that content.)
(I thought I'd clarify this, because this comment got downvoted - possibly because the downvoter misunderstood it as sarcasm?)
bahahahaha
Nah, the wiki makes it much easier.
-- Tim Evans-Ariyeh
Ideally, it would be nice if the world can move towards caring about the full outcome over factors like the satisfication of baseline levels of effort in more and more situations, not just exceptional ones.
Myers, D. G. (2012). Exploring social psychology (6th ed.). New York: McGraw-Hill, P.334.
So basically: be close to friends and family, save some money, find a job you're good at.
That's close to my understanding of the quote. I suppose, "autonomy" means not just financial independence, but the sense of inner self, something beyond social roles.