= 27d567bc2d48d9f40e9ccc20ea370b15)
Rationality Quotes September 2013
Another month has passed and here is a new rationality quotes thread. The usual rules are:
- Please post all quotes separately, so that they can be upvoted or downvoted separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)
- Do not quote yourself.
- Do not quote from Less Wrong itself, HPMoR, Eliezer Yudkowsky, or Robin Hanson. If you'd like to revive an old quote from one of those sources, please do so here.
- No more than 5 quotes per person per monthly thread, please.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
Comments (456)
Bizarro Blog
Sorry, this is nonsense. It's not hard to Google up a copy of the FCC rules. http://www.fcc.gov/guides/obscenity-indecency-and-profanity :
I am fairly sure that "I’m going to rape your 8-year-old daughter with a trained monkey" would count as describing sexual activities in patently offensive terms, and would not be allowed when direct use of swear words would not be allowed. Just because you don't use a list of words doesn't mean that what you say will be automatically allowed.
Furthermore, the Wikipedia page on the seven words ( http://en.wikipedia.org/wiki/Seven_dirty_words ) points out that " The FCC has never maintained a specific list of words prohibited from the airwaves during the time period from 6 a.m. to 10 p.m., but it has alleged that its own internal guidelines are sufficient to determine what it considers obscene." It points out cases where the words were used in context and permitted.
In other words, this quote is based on a sound-bite distortion of actual FCC behavior and as inaccurate research, is automatically ineligible to be a good rationality quote.
What is the basis for you being sure?
Howard Stern, a well-known "shock jock" spent many years on airwaves regulated by the FCC. He more or less specialized in "describing sexual activities in patently offensive terms" and while he had periodic run-ins with the FCC, he, again, spent many years doing this.
The FCC rule is deliberately written in a vague manner to give the FCC discretionary power. As a practical matter, the seven dirty words are effectively prohibited by FCC and other offensive expressions may or may not be prohibited. Broadcasters occasionally test the boundaries and either get away with it or get slapped down.
Yes, and this illustrates another problem: if we agreed on what to ban, it would make more sense to use discretionary human judgment than rules which might be manipulated or Munchkin-ed. We don't agree.
I do think it would make sense in the abstract to ban speech if we had scientific reason to think it harmed people, the way we had reason to think leaded gasoline harmed people in the 1920s. But I only know one class of speech where that might apply, and it'll never get on TV anyway. ^_^
The reason is that banning certain words works much better as a Schelling point.
I don't buy it. Even if we accept for the sake of argument that limiting sexual references on broadcast TV is a good plan (a point that I don't consider settled, by the way), using dirty words as a proxy runs straight into Goodhart's law: the broadcast rules are known in advance, and innuendo's bread and butter to TV writers. A good Schelling point has to be hard to work around, even if you can't draw a strict line; this doesn't qualify.
Better for what, and better than what alternatives?
You wind up in endless arguments about whether this particular show is beyond the pail.
That doesn't seem like it answers my question.
What's the goal in this case? This sounds like it's only attempting to address effectiveness at avoiding disputes over standards, but that could more easily be achieved by not having any restrictions at all.
-- Tim Evans-Ariyeh
Ideally, it would be nice if the world can move towards caring about the full outcome over factors like the satisfication of baseline levels of effort in more and more situations, not just exceptional ones.
Jon Stewart, talking to Richard Dawkins (S18, E156)
Let's get one thing straight: ignorance killed the cat.
Curiosity was framed.
Myers, D. G. (2012). Exploring social psychology (6th ed.). New York: McGraw-Hill, P.334.
So basically: be close to friends and family, save some money, find a job you're good at.
That's close to my understanding of the quote. I suppose, "autonomy" means not just financial independence, but the sense of inner self, something beyond social roles.
--"Adventure Time" episode "The Businessmen": the zombie businessmen are explaining why they are imprisoning soft furry creatures in a glass bowl.
Saturday Morning Breakfast Cereal
G K Chesterton
I don't think that's the case. There are plenty of shy intellectuals who don't push their ideas on other people. Darwin sat more than a decade on his big idea.
There are ideas that are about qualia. It doesn't make much sense to try to explain a blind person what red looks like and the same goes for other ideas that rest of observed qualia instead of resting on theory. If I believe in a certain idea because I experienced a certain qualia and I have no way of giving you the experience of the same qualia, I can't explain you the idea. In some instances I might still try to explain the blind what red looks like but there are also instance where I see it as futile.
One way of teaching certain lessons in buddhism is to give a student a koan that illustrates the lesson and let him meditate over the koan for hours. I don't see anything dishonest about teaching certain ideas that way.
If someone thinks about a topic in terms of black and white it just takes time to teach him to see various shades of grey.
Rationality wakes up last:
Scott Adams on waking up with a numb arm.
I woke up one time with both arms completely numb. I tried to turn the light on and instead fell out of bed. I felt certain that I was going to die right then.
Never experienced this exact experience - I don't sleep on my arm - but waking up stupid? Definitely.
Odd, this has never happened to me. Not the experience of waking up with a numb arm, but the experience of being at all worried about it.
I was quite worried the first time I experienced a numb arm which was both completely dead to sensation and totally immobile for multiple minutes, but after that had happened before, successive occurrences were no longer particularly worrying.
I've experienced 'pins and needles' many times, but a totally 'dead' arm only once. I didn't have any control over it, and when I tried to move it I hit myself in the nose. Quite hard, too!
When I experienced a "totally dead arm," I didn't just not have control over it, I couldn't even wiggle my fingers. It was pretty frightening, since as far as I knew the arm might have experienced extensive cell death from blood deprivation; after all, I had no sign of it being operational at all. My circulation was poor enough that I couldn't even tell if it was still warm, beyond residual heat from my lying on it.
It's happened twice again since then though, and the successive occasions were not particularly distressing.
IIRC the numbness is caused by nerve compression, not blood-flow cutoff.
edit: Apparently it can be either way: http://www.wisegeek.org/what-are-the-most-common-causes-of-numbness-while-sleeping.htm
edit2: And another source claims it's due to nerves, so I dunno. I do find the nerve explanation more plausible than the blood-flow one.
I couldn't do anything with the arm either, it just felt as if it wasn't there. It was decades ago, but I think I used my shoulder muscles to try and move it. I was probably scared too, but that part of the memory is quite vague.
Richard Mitchell - Less Than Words Can Say
Counterexample: "it".
That seems to support the quote, actually: "it" typically has a single antecedent, or a small enough set that the correct antecedent can be easily identified by context. When it cannot be identified by context, this is seen as a writing error (such as here, here, or here).
Possible counterexamples (there are probably better ones):
All of these are dummy subjects. English does not allow a null anaphor in subject position; there are other languages that do. ("There", in that last clause, was also a dummy pronoun.)
For each of those, the meaning of "it" is clear from context. If it weren't, then it would be uncommunicative writing.
What is the intended lesson or rationality insight here?
-- John Scalzi
So is the failure mode of many people who are not, and don't hold themselves to be, clever. I fail to see the correlation.
ETA: Scalzi addresses a very specific topic, and even then he really seems to address some specific anecdote that he doesn't share. I don't think it's a rationality quote.
Mark Crislip - Science-Based Medicine
Reality cares about your beliefs.
People who don't believe in ego depletion don't get as much ego depleted as people who do believe in it.
People who believe that stress is unhealthy have a higher mortality when they have high stress than people who don't hold that belief.
I would expect that if you have more ego depletion than other people it would result in you being more likely to believe in ego depletion. Similarly, if you're suffering health problems due to stress, it would make you think stress is unhealthy.
Your point still stands. Reality does care about your beliefs when the relevant part of reality is you.
I'd guess that there is causation in both directions to some extent, leading to a positive feedback loop.
How high is your confidence that the effect can be completely explained that way?
Not that high, but it does throw into question any studies showing a correlation, and it seems strange to site an example there's no evidence for.
What does that mean in numbers?
The placebo effect has little relevant effect. People who believe they can fly don't fare better when pushed of cliffs. A world where you believe x is different from a world where you believe not-x, and that has slight physical effects given that we are embodied, but to say 'Reality cares about your beliefs' sounds far to much like a defence of idealism, or the idea that 'everyone has their own truths'.
I'm not sure whether that's true, the last time I investigated that claim I don't found the evidence compelling. Placebo's are also a relatively clumsy way of changing beliefs intentionally.
How do you know? If you pick a height that kills 50% of the people who don't believe that they can fly, I'm not sure that the number of people killed is the same for those who hold that belief. The belief is likely to make people more relaxed when they are pushed over the cliff which is helpful for surviving the experience.
I doubt that you find many people who hold that belief with the same certainity that they believe the sun rises tomorrow. If you don't like idealism, argue based on the beliefs that people actually hold in reality instead of escaping into thought experiments.
I would call 20000 death Americans per year for the belief that stress is unhealthy more than a slight physical effect.
I don't think that the fact that you pattern match it that way speaks against the idea. I think the original quote comes from a place of Descartes inspired mind-body dualism. We are embodied and the content of our mind has effects.
The original quote is taken from an article about the vaccine controversy. People who don't vaccinate because they believe that God will protect them or whatever actually exist, and they may be slightly less likely to fall ill than people who don't vaccinate but don't hold that belief but a lot more likely to fall ill than people who do vaccinate.
I think that there are few Christians who believe that no Christian who doesn't vaccinate will get Measles.
Many Chrisitan's do believe that there's evil in the world. They believe that for some complicated reason that they don't understand sometimes God will allow evil to exist. God is supposed to be an agent who's actions a mere human can't predict with certainity.
According to that frame, if someone get's Measles it's because God wanted that to happen. If on the other hand a child dies because of adverse reaction of a vaccine that the doctor gave the child then the parent shares responsibility for that harm because he allowed the vaccination.
I also don't now how the example of Japan is supposed to convince any Christian that his supposed belief in God preventing Measles of believing Christians is wrong.
While we are at the topic of the effect of beliefs, I don't think there good research about how beliefs that people hold effect whether they get illnesses. Part of the reason is that most doctors who do studies about the immune system don't think that beliefs are in their domain because they study the body and not the mind.
parodie
Richard Mitchell - Less Than Words Can Say
His Master's Voice, Stanislaw Lem; p. 106 from the Northwestern University Press 3rd edition, 1999
I like the self-test idea, but this sort of defeatism is kind of, well, self-defeating.
I think it's true. Short of crude measures like stimulants, it does seem to ebb and flow for no obvious reasons. And it's useful to know if you're currently in a doldrum - you can give up forcing yourself to try to work on creative material, and turn to all the usual chores and small tasks that build up.
For fairness sake.
From http://metamodern.com/2009/05/17/how-to-understand-everything-and-why/
I don't think that's true. I can learn that I feel better when I'm exposed to sunlight without knowing the in- and outs- of vitamin D biochemistry.
The things that matters is to accurately measure whether I feel better and to measure when I'm exposed to sunlight.
When you know a thing, to hold that you know it, and when you do not know a thing, to allow that you do not know it. This is knowledge.
Confucius, Analects
Black Books, Elephants and Hens. H/t /u/mrjack2 on /r/hpmor.
-- Richard James, founding priest of a Toronto based Wicca church, quoted in a thegridto article
by Hannes Leitgeb, from his joint teaching course with Stephan Hartmann (author of Bayesian Epistemology) on Coursera entitled 'An Introduction to Mathematical Philoosphy'.
The course topics are "Infinity, Truth, Rational Belief, If-Then, Confirmation, Decision, Voting, and Quantum Logic and Probability". In many ways, a very LW-friendly course, with many mentions and discussions of people like Tarski, Gödel etc.
The biggest problem in the world is too many words. We should be able to communicate, distribution graphs of past experiences, directly from one human brain to another. ~Aang Jie
Theophanis the Monk, "The Ladder of Divine Grace"
Nick Szabo
Is this a similar message to Penn Jillette saying:
"If you don’t pay your taxes and you don’t answer the warrant and you don’t go to court, eventually someone will pull a gun. Eventually someone with a gun will show up. "
or did I miss the boat?
Well, it's similar, but for two differences:
1) It uses a different and wider category of examples. Viz. "initiate force [...] to compel them to hand over goods, to let us search their property, or to testify."
2) It makes a consequentialist claim about forcing people to e.g. let us search their property for evidence: "we can't properly respond to a global initiation of force without local initiations of force."
The second difference here is important because it directly contradicts the typical libertarian claim of "if we force people to do things much less than we currently do, that will lead to good consequences." The first difference is rhetorically important because it is a place where people's gut reaction is more likely to endorse the use of force, and people have been less exposed to memes about forcibly searching peoples' property (compared to the ubiquity of people disliking taxes) that would cause them to automatically respond rather than thinking.
Actually that isn't what Szabo is saying. His point is to contradict the claim of the anarcho-capitalists that "if we never force people to do things, that will lead to good consequences."
From Obvious Adam, a business book published in 1916.
Slightly edited from Scott Adams' blog.
And a similar sentiment from SMBC comics.
I personally can't see how a monkey turns into a human. But that's irrelevant because that is not the claim of natural selection. This makes a strawman of most positions that endorse something approximately like free will. Also:
Just the legal system? Gah. Everybody on earth does this about 200 times a day.
I don't think that most positions that endorse free will don't believe at all that evolution happens.
When it comes to contempory philosophers I think a minority of those who advocate for the existence of free will deny evolution.
I know. I was making an analogy between a strawman of NS and a strawman of free will. Please read the "this" in "This makes a strawman" as referring to the OP.
Roger Ebert
Would be nice if this were true.
It's probably true for academic film theory. I mean how hard could it really be?
-- Norman Page, Auden and Isherwood: The Berlin Years
Not quite seeing this as a rationality quote. What's your reasoning?
"The Great Phrase-book Fallacy" is both amusing and instructive. I laughed when I read it because I remembered I'd been a victim of it too once, in less seedier circumstances.
But it's not who you are underneath, it's what you do that defines you.
-Rachel Dawes, Batman Begins
Q: Why are Unitarians lousy singers? A: They keep reading ahead in the hymnal to see if they agree with it.
— Jack Vance, The Languages of Pao
Improbable would seem more appropriate.
Friedrich Nietzsche
Because it's really really useful?
Jeez, people really don't appreciate poetic language around here, huh?
(That would probably be close to my answer too, I'm just a little stunned by all the downvotes.)
-- Gordon R. Dickson, "The Tactics of Mistake".
Anonymous, found written in the Temple at 2013 Burning Man
Part of that seems to be from HPMOR. I'm not sure where the rest comes from.
Yeah, almost certainly HPMOR inspired. Eliezer's work has spread far.
Eugene McCarthy, Human Origins: Are We Hybrids?
As a non-biologist, I kind-of suspect that article is supposed to be some kind of elaborate joke. It sounds convincing to me, but then again, so did Sokal (1996) to non-physicists; my gut feelings' prior probability for that claim is tiny (but probably tinier than rationally warranted; possibly, because it kind-of sounds like a parody of ancient astronaut hypotheses); and I can't find any mention of any mammal inter-order hybrids on Wikipedia.
This is a blatant parody. Probability of pig+chimp hybrids involved in human origins are at pascal-low levels.
This is worthy of notice. It really shouldn't have been remotely convincing..
Can you identify the factors which caused you to give the statements in this article more credibility than you would have given to any random internet source of an unlikely-sounding claim? Information about what went wrong here might be useful from a rationality-increasing perspective.
Mostly, the fact that I don't know shit about biology, and the writer uses full, grammatical sentences, cites a few references, anticipates possible counterarguments and responds to them, and more generally doesn't show many of the obvious signs of crackpottery.
This is exactly why I (amongst many?) find it so hard to separate the good-stuff from the bad-stuff. It's the way the matter is brought to you, not the matter itself. Very thoughtful way of bringing it, as Army1987 says, references, anticipation of counterarguments etc.
I would also very wary of McCarthy arguement. As having studied bioinformatics myself I would say:
Show me the human genes that you think come from pigs. If you name specific genes we can run our algorithms. Don't talk about stuff like the form vertebra when we have sequenced the genomes.
Sokal's paper brought up the possibility of a morphogenetic field affecting quantum mechanics, which sounds slightly less rigorous than a Discworld joke -- Sir Pratchett can at least get the general aspects of quantum physics correctly. Likewise, Mrs. Jenna Moran's RPGs have more meaningful statements on set theory than Sokal's joking conflation of the axiom of equality and feminist/racial equality. I'd expect a lot of non-physicists would consider it unconvincing, especially if you allow them the answer "this paper makes no sense".
((I'd honestly expect false positives, more than false negatives, when asking average persons to /skeptically/ test papers on quantum mechanics for fraud. Thirty pages of math showing a subatomic particle to be charming has language barrier problems.))
The greater concern here is that the evidence Mr. McCarthy uses to support his assertions is incredibly weak. The vast majority of his list of interspecies hybrids, for example, are either intra-familiae or completely untrustworthy (some are simply appeals to legends or internet hoax, like the cabbit or dog-bear hybrids). The only example of remotely similar variation to a chimpanzee-pig hybrid while being remotely trustworthy is an alleged rabbit-rat cross, but chasing the citation shows that the claimed evidence likely had a different (and at the time of the original experiment, unknown) cause and that the fertilization never occurred. Other cases conflate mating behavior and fertility, by which definition humans should be capable of hybridizing with rubber and glass. The sheer number of untrustworthy citations -- and, more importantly, that they're mixed together with the verifiable and known good ones -- is a huge red flag.
The quote's interesting -- and correct! as anyone who's shown the double-slit experiment can show -- but there's probably better ways to say it and theories to associate it with.
The concept doesn't come from Sokal but from Rupert Sheldrake who used the term in his 1995 book (http://www.co-intelligence.org/P-morphogeneticfields.html).
There are plenty of New Age people who seriously believe that the world works that way.
Or find it a reasonable / plausible theory... I'm married to one who evolved into one who reads that pseudo-science, instead of the Stephen Hawking she used to read 20 years ago...
Yeah, it's a good quote promoting open-mindedness, but of course that's because crackpots spend a lot of time trying to hide their theories from any criticism in the name of open-mindedness.
Another good one from the same source:
John LeCarre, explaining that he didn't have insider information about the intelligence community, and if he had, he would not have been allowed to publish The Spy Who Came in from the Cold, but that a great many people who thought James Bond was too implausible wanted to believe that LeCarre's book was the real deal.
Scott Adams
Ted Chiang, Tower of Babylon
A. P. Herbert, Uncommon Law.
Caution in applying such a principle seems appropriate. I say this because I've long since lost track of how often I've seen on the Internet, "I lost all respect for X when they said [perfectly correct thing]."
I don't lose all respect for X based on one thing they say, but I do increase my respect in them if the controversial or difficult things they say are correct and I conserve expected evidence.
I agree. It strengthens your point to note that, although the quote is normally used seriously, the author intended it mischievously. In context, the "thirteenth stroke" is a defendant, who has successfully rebutted all the charges against him, making the additional claim that "this [is] a free country and a man can do what he likes if he does nobody any harm."
This "crazy" claim convinces the judge to convict him anyway.
For most people, is it necessarily wrong to lose all respect for someone in response to a true statement? Most people are respecting things other than truth, and the point "anyone respectable would have known not to say that" can remain perfectly valid.
A. P. Herbert, Uncommon Law.
Jeremy Silman
Oglaf (Original comic NSFW)
How have I been reading Oglaf for so long without knowing about the epilogues?
... the what.
Ahh I just finished that.
For anyone unaware, SMBC has an additional joke panel when you mouse over the red button at the bottom
Actually, you have to click it now. Just a heads up to anyone reading this and trying to find them.
AAAARGH!!! Why do they keep it secret?
That's almost as annoying as that you have to know the name of Zach's wife to create an account and comment, when for a long time the name of Zach's wife was not findable either on the website or via Google.
(I don't remember her name.)
Thank you very much.
I ... was not aware it was even possible to comment on SMBC.
It's an example of failing to update traditions after their original purpose has eroded, for the record. It was originally a reward for voting, which is why SMBC fans still refer to it as a "votey". The voting atrophied, while creating the reward became part of his routine.
Kelly Weinersmith.
It's usually the funnest panel, too.
...oh crap, I'm going to have to reread the whole thing, aren't I.
And the mouseovers. And the alt text, which is different again.
And the mock ads at the bottom.
ETA: Explanation: Sometimes the banner at the bottom will contain an actual (randomized) ad, but many of the comics have their own funny mock ad associated. (When I noticed this, I went through all the ones I had already read again, to not miss out on that content.)
(I thought I'd clarify this, because this comment got downvoted - possibly because the downvoter misunderstood it as sarcasm?)
What's the difference between a mouseover and an alt text?
Mouseover is javascript EDIT: or CSS and shows up when you hover your pointer over some trigger area. Alt text is plain HTML and shows up when the image (or whatever it is alt text for) doesn't load.
Javascript is not actually required. CSS handles it.
No, mouseover is TITLE= and alt text ls ALT=. Mouseover doesn't rely on Javascript. Alt text is specifically for putting in place of an image; it used to be used for mouseovers as well, but then TITLE= came in for that.
How do you get alt text to appear if the image loads? Read source?
Yup.
There's also title text (often called a tool tip) which appears when you hover the mouse over an image, but is a plain HTML feature.
bahahahaha
Nah, the wiki makes it much easier.
"To know thoroughly what has caused a man to say something is to understand the significance of what he has said in its very deepest sense." -Willard F. Day
On the other hand, one should consider not only what was said, but also what should have been said.
Peter Shor replying in the comment section of Scott Aaronson's blog post Firewalls.
-Leonard Susskind, Susskind's Rule of Thumb
Great quote.
Unfortunately, we find ourselves in a world where the world's policy-makers don't just profess that AGI safety isn't a pressing issue, they also aren't taking any action on AGI safety. Even generally sharp people like Bryan Caplan give disappointingly lame reasons for not caring. :(
Why won't you update towards the possibility that they're right and you're wrong?
This model should rise up much sooner than some very low prior complex model where you're a better truth finder about this topic but not any topic where truth-finding can be tested reliably*, and they're better truth finders about topics where truth finding can be tested (which is what happens when they do their work), but not this particular topic.
(*because if you expect that, then you should end up actually trying to do at least something that can be checked because it's the only indicator that you might possibly be right about the matters that can't be checked in any way)
Why are the updates always in one direction only? When they disagree, the reasons are "lame" according to yourself, which makes you more sure everyone's wrong. When they agree, they agree and that makes you more sure you are right.
It's not so much that I'm a better truth finder, it's that I've had the privilege of thinking through the issues as a core component of my full time job for the past two years, and people like Caplan only raise points that have been accounted for in my model for a long time. Also, I think the most productive way to resolve these debates is not to argue the meta-level issues about social epistemology, but to have the object-level debates about the facts at issue. So if Caplan replies to Carl's comment and my own, then we can continue the object-level debate, otherwise... the ball's in his court.
This doesn't appear to be accurate. E.g. Carl & Paul changed my mind about the probability of hard takeoff. And when have I said that some public figure agreeing with me made me more sure I'm right? See also my comments here.
If I mention a public figure agreeing with me, it's generally not because this plays a significant role in my own estimates, it's because other people think there's a stronger correlation between social status and correctness than I do.
Yes, but why Caplan did not see it fit to think about the issue for a significant time, and you did?
There's also the AI researchers who have had the privilege of thinking about relevant subjects for a very long time, education, and accomplishments which verify that their thinking adds up over time - and who are largely the actual source for the opinions held by the policy makers.
By the way, note that the usual method of rejection of wrong ideas, is not even coming up with wrong ideas in the first place, and general non-engagement of wrong ideas. This is because the space of wrong ideas is much larger than the space of correct ideas.
What I expect to see in the counter-factual world where the AI risk is a big problem, is that the proponents of the AI risk in that hypothetical world have far more impressive and far more relevant accomplishments and credentials.
The first problem with highly speculative topics is that great many arguments exist in favour of either opinion on a speculative topic. The second problem is that each such argument relies on a huge number of implicit or explicit assumptions that are likely to be violated due to their origin as random guesses. The third problem is that there is no expectation that the available arguments would be a representative sample of the arguments in general.
Hmm, I was under the impression that you weren't a big supporter of the hard takeoff to begin with.
Well, your confidence should be increased by the agreement; there's nothing wrong with that. The problem is when it is not balanced by the expected decrease by disagreement.
There are a great many differences in our world model, and I can't talk through them all with you.
Maybe we could just make some predictions? E.g. do you expect Stephen Hawking to hook up with FHI/CSER, or not? I think... oops, we can't use that one: he just did. (Note that this has negligible impact on my own estimates, despite him being perhaps the most famous and prestigious scientist in the world.)
Okay, well... If somebody takes a decent survey of mainstream AI people (not AGI people) about AGI timelines, do you expect the median estimate to be earlier or later than 2100? (Just kidding; I have inside information about some forthcoming surveys of this type... the median is significantly sooner than 2100.)
Okay, so... do you expect more or fewer prestigious scientists to take AI risk seriously 10 years from now? Do you expect Scott Aaronson and Peter Norvig, within 25 years, to change their minds about AI timelines, and concede that AI is fairly likely within 100 years (from now) rather than thinking that it's probably centuries or millennia away? Or maybe you can think of other predictions to make. Though coming up with crisp predictions is time-consuming.
Well, I too expect some form of something that we would call "AI", before 2100. I can even buy into some form of accelerating progress, albeit the progress would be accelerating before the "AI" due to the tools using relevant technologies, and would not have that sharp of a break. I even do agree that there is a certain level of risk involved in all the future progress including progress of the software.
I have a sense you misunderstood me. I picture this parallel world where legitimate, rational inferences about the AI risk exist, and where this risk is worth working at in 2013 and stands out among the other risks, as well as any other pre-requisites for making MIRI worthwhile hold. And in this imaginary world, I expect massively larger support than "Steven Hawkins hooked up with FHI" or what ever you are outlining here.
You do frequently lament that the AI risk is underfunded, under-supported, and there's under-awareness about it. In the hypothetical world, this is not the case and you can only lament that the rational spending should be 2 billions rather than 1 billion.
edit: and of course, my true rejection is that I do not actually see rational inferences leading there. The imaginary world stuff is just a side-note to explain how non-experts generally look at it.
edit2: and I have nothing against FHI's existence and their work. I don't think they are very useful, or address any actual safety issues which may arise, though, but with them I am fairly certain they aren't doing any harm either (Or at least, the possible harm would be very small). Promoting the idea that AI is possible within 100 years, however, is something that increases funding for AI all across the board.
Right, this just goes back to the same disagreement in our models I was trying to address earlier by making predictions. Let me try something else, then. Here are some relevant parts of my model:
Luke, why are you arguing with Dmytry?
The question should not be about "highly credentialed" people alone, but about how they fare compared to people who are rather very low "credentialed".
In particular, on your list, I expect people with fairly low credentials to fare much worse, especially at identification of the important issues as well as on rational thinking. Those combine multiplicatively, making it exceedingly unlikely - despite the greater numbers of the credential-less masses - that people who lead the work on an important issue would have low credentials.
What's EA? Effective altruism? If it's an existential risk, it kills everyone, selfishness suffices just fine.
Ohh, come on. That is in no way a demonstration that insurance companies in general follow faulty strategies, and especially is not a demonstration that you could do better.
Indeed.
A selfish person protecting against existential risk builds a bunker and stocks it with sixty years of foodstuffs. That doesn't exactly help much.
No doubt! I wasn't comparing highly credentialed people to low-credentialed people in general. I was comparing highly credentialed people to Bostrom, Yudkowsky, Shulman, etc.
After reading Robin's exposition of Bryan's thesis, I would disagree that his reasons are disappointingly lame.
Which could either indicate that the reasons are good or that your standards are lower than Luke's and so trigger no disappointment.
Bryan is expressing a "standard economic intuition" but... did you see Carl's comment reply on Caplan's post, and also mine?
I did see Eelco Hoogendoorn 's and it is absolutely spot on.
I'm hardly a fan of Caplan, but he has some Bayesianism right:
Based on how things like this asymptote or fail altogether, he has a low prior for foom.
He has low expectation of being able to identify in advance (without the work equivalent to the creation of the AI) exact mechanisms by which it is going to asymptote or fail, irrespective of whenever it does or does not asymptote or fail, so not knowing such mechanisms does not bother him a whole lot.
Even assuming he is correct he expects a plenty of possible arguments against this position (which are reliant on speculations), as well as expects to see some arguers, because the space of speculative arguments is very huge. So such arguments are not going to move him anywhere.
People don't do that explicitly any more than someone who's playing football is doing Newtonian mechanics explicitly. Bayes theorem is no less fundamental than the laws of motion of the football.
Likewise for things like non-testability: nobody's doing anything explicitly, it is just the case that due to something you guys call "conservation of expected evidence" , when there is no possibility of evidence against a proposition, then a possibility of evidence in favour of the proposition would violate the Bayes theorem.
I'm not sure how you could have such a situation, given that absence of expected evidence is evidence of the absence. Do you have an example?
Well, the probabilities wouldn't be literally zero. What I mean is that lack of a possibility of strong evidence against something, and only a possibility of very weak evidence against it (via absence of evidence) implies that strong evidence in favour of it must be highly unlikely. Worse, such evidence just gets lost among the more probable 'evidence that looks strong but is not'.
Ah, I think I follow you.
Absence of evidence isn't necessarily a weak kind of evidence.
If I tell you there's a dragon sitting on my head, and you don't see a dragon sitting on my head, then you can be fairly sure there's not a dragon on my head.
On the other hand, if I tell you I've buried a coin somewhere in my magical 1cm deep garden - and you dig a random hole and don't find it - not finding the coin isn't strong evidence that I've not buried one. However, there there's so much potential weak evidence against. If you've dug up all but a 1cm square of my garden - the coin's either in that 1cm or I'm telling porkies, and what are the odds that - digging randomly - you wouldn't have come across it by then? You can be fairly sure, even before digging up that square, that I'm fibbing.
Was what you meant analogous to one of those scenarios?