Comment author: 01 December 2013 08:18:05PM 8 points [-]

Amusingly, google chrome autofill still remembered my answers from last year. This made filling the demographic part a bit faster, and allowed a little game: after giving a probability estimation I could check my answer from a year ago.

Comment author: 27 September 2013 02:27:29AM 4 points [-]

I like this. For example, a human holding a white mouse in one hand and looking at it.

The problem, though, is that once you have a human in the shot it's real hard to get someone looking at the book for the first time to identify with the white mouse. One strategy is to de-humanize the human - like maybe only have a human eye filling up the background, with the shot centered on a butterfly. example, sort of. Eh, seems pretty difficult now that I think about it more.

Comment author: 28 September 2013 10:10:00PM 1 point [-]

The smaller thing could be a human, too. Giant, good looking but creepy child holding small vulnerable human in one hand, looking at it emotionlessly. But MIRI will not like this version, because they really want to avoid anthropomorphizing the AI.

Comment author: 18 September 2013 11:13:32PM 14 points [-]

Looking at the individual level, most of us had close friends who had lost 30 years of potential life.

Suppose you could either extend the life of one close friend by 30 years, or the lives of all of your friends by 10 years. (Hopefully you have more than three friends.) Page is pointing out that the second could possibly be on the table, but it wouldn't be obvious because we're so used to treating rare serious diseases instead of making everyone a bit healthier or live a bit longer on the margins.

Comment author: 19 September 2013 02:08:21AM 2 points [-]

I fully agree with this point, and I fully agree with Page's goals. But I think there are things here that a simple total-years-of-potential-life-lost framework can not capture. As you might have guessed even from my first comment, this issue is very personal to me. Not long ago a good friend died after terrible suffering, leaving three young children behind. That's very sad, and I really don't know for what values of N could this be balanced in a utilitarian sense by lengthening the healthy old age of N of my friends with 10 years. Obviously, such trade-offs are taboo, but even if I try to force myself into some detached outside view, I still believe that number N must be large.

Comment author: 18 September 2013 08:39:18PM 3 points [-]

“Are people really focused on the right things? One of the things I thought was amazing is that if you solve cancer, you’d add about three years to people’s average life expectancy,” Page said. “We think of solving cancer as this huge thing that’ll totally change the world. But when you really take a step back and look at it, yeah, there are many, many tragic cases of cancer, and it’s very, very sad, but in the aggregate, it’s not as big an advance as you might think.” (Larry Page as quoted in the Time article)

This is something like the ecological fallacy. In the aggregate, we lose 3 years of potential life because of cancer. No big deal. Looking at the individual level, most of us had close friends who had lost 30 years of potential life.

Comment author: 29 June 2013 08:05:24PM *  4 points [-]

Alright, I will take your word for it. I had never seen anyone say that the classical Newtonian-mechanical sort of n-body problem was almost identical to a quantum intra-atomic version, though.

There are differences and complications because of things like Uncertainty, magnetism, and the Pauli exclusion principle, but to first order the dominant effect on an individual atomic particle is the Coulomb force and the form of that is identical to the Gravitational force. The symmetry in the force laws may be more obvious than the Hamiltonian formulation I gave before.

$F_G=\frac{Gm_1m_2}{r^2} \: and \: F_C=\frac{k_eq_1q_2}{r^2}$

The particularly interesting point is that even without doing any quantum mechanics at all, even if atomic bonding were only a consequence of classical electrostatic forces, we still wouldn't be able to solve the problem. The difficulty generated by the n-body problem is in many ways much greater than the difficulty generated by quantum mechanics.

Also, nice summary.

Comment author: 02 July 2013 10:57:36AM 3 points [-]

I am not a physicist, but this stack exchange answer seems to disagree with your assessment: What are the primary obstacles to solve the many-body problem in quantum mechanics?

Comment author: 15 June 2013 05:30:57PM 10 points [-]

Someone write a script to sort comments in Rationality Quotes threads by net karma per character!

Comment author: 18 June 2013 01:44:50AM *  7 points [-]

I did exactly that after looking at this thread, and only spotted your comment when I wanted to post the results.

I skipped some obvious refinements as this was a 5 minute project.

• 55 A Bet is a Tax on Bullshit. Alex Tabarrok
• 45 Luck is statistics taken personally. Penn Jellete
• 33 Comic Quote Minus 37 -- Ryan Armand Also a favourite.
• 34 Nobody is smart enough to be wrong all the time.Ken Wilber
• 32 A problem well stated is a problem half solved.Charles Kettering
• 48 I will not procrastinate regarding any ritual granting immortality. --Evil Overlord List #230
• 29 The greatest weariness comes from work not done.-Eric Hoffer
• 26 "Most haystacks do not even have a needle." -- Lorenzo
• 24 "I accidentally changed my mind." my four-year-old
• 31 Doubt is not a pleasant condition, but certainty is absurd. Voltaire
• 39 People say "think outside the box," as if the box wasn't a fucking great idea.Sean Thomason
• 38 The Noah principle: predicting rain doesn’t count, building arks does. -Warren E. Buffett
• 37 It’s easy to lie with statistics, but it’s easier to lie without them. -Fred Mosteller
• 30 If one does not know to which port one is sailing, no wind is favorable.-Seneca
• 34 It is the mark of a truly intelligent person to be moved by statistics.George Bernard Shaw
• 34 "Working in mysterious ways" is the greatest euphemism for failure ever devised.TheTweetOfGod
• 12 Death is the gods' crime. Unsounded
• 24 The most practical thing in the world is a good theory. Helmholtz
• 29 When the facts change, I change my mind. What do you do, sir? John Maynard Keynes
• 28 Writing program code is a good way of debugging your thinking. -- Bill Venables
• 28 It is easy to be certain....One has only to be sufficiently vague.Charles S. Peirce
• 30 Truth is much too complicated to allow anything but approximations. — John Von Neumann
• 31 There is one rule that's very simple, but not easy: observe reality and adjust. Ran Prieur
• 21 Things are only impossible until they're not. -- Jean-Luc Picard
• 24 Part of the potential of things is how they break. Vi Hart, How To Snakes
• 25 A scholar is just a library’s way of making another library. Daniel Dennett
• 29 We should be careful to get out of an experience only the wisdom that is in it. Mark Twain
• 29 The Company that needs a new machine tool is already paying for it. -old Warner Swasey ad
• 25 "Any sufficiently analyzed magic is indistinguishable from SCIENCE!" ~Girl Genius
• 26 Complex problems have simple, easy to understand wrong answers. — Grossman's Law
• 22 Most people would rather die than think; many do. – Bertrand Russell
• 22 The only road to doing good shows, is doing bad shows.Louis C.K., on Reddit
• 28 My brain technically-not-a-lies to me far more than it actually lies to me.-- Aristosophy (again)
• 23 The truth will set you free. But first, it will piss you off. Gloria Steinem
• 27 Nature draws no line between living and nonliving. -- K. Eric Drexler, Engines of Creation
• 22 It is better to destroy one's own errors than those of others. Democritus
• 12 Reality is not optional. Thomas Sowell
• 17 Statistics is applied philosophy of science. A. P. Dawid
• 22 Forget Jesus. The stars died so that you could be here today. Lawrence Krauss
• 23 Go down deep enough into anything and you will find mathematics. Dean Schlicter
• 24 We are built to be effective animals, not happy ones. -Robert Wright, The Moral Animal
• 19 Being right too soon is socially unacceptable. Robert A. Heinlein
• 14 "Anything you can do, I can do meta" -Rudolf Carnap
• 17 Mind is a machine for jumping to conclusions - Daniel Kahneman
• 26 A faith which cannot survive collision with the truth is not worth many regrets.Arthur C. Clarke
• 26 Nobody panics when things go "according to plan"… even if the plan is horrifying. The Joker
• 23 "Convictions are more dangerous enemies of truth than lies." --Friedrich Nietzsche
• 20 I honestly don't know. Let's see what happens. -- Hans. The Troll Hunter
• 16 Luck is opportunity plus preparation plus luck.--Jane Espenson
• 20 The singularity is my retirement plan. -- tocomment, in a Hacker News post
• 19 Better our hypotheses die for our errors than ourselves. -- Karl Popper
• 22 Errors using inadequate data are much less than those using no data at all.-Charles Babbage
• 19 In general, we are least aware of what our minds do best. — Marvin Minsky
• 20 It is easier to love humanity than to love one's neighbor.--Eric Hoffer, on Near/Far
• 18 "If God gives you lemons, you find a new God."-- Powerthirst 2: Re-Domination
• 17 Truth comes out of error more easily than out of confusion.-Francis Bacon
• 23 Opening your eyes doesn't make a bad picture worse. http://onefte.com/2011/07/17/bully-for-you/
• 19 Know the hair you have to get the hair you want. -Pantene Pro-V hair care bottle
• 16 The best way to escape from a problem is to solve it. -Alan Saporta
• 16 Not to know is bad; not to wish to know is worse. — Wolof proverb
• 20 Those who can make you believe absurdities can make you commit atrocities. -- Voltaire
Comment author: 13 June 2013 10:56:46PM 5 points [-]

Even I think they'd take a mind upload seriously - that might really produce a huge public update though probably not in any sane direction - though I don't expect that to happen before a neuromorphic UFAI is produced from the same knowledge base. They normatively ought to take a spider upload seriously. Something passing a restricted version of a Turing test might make a big public brouhaha, but even with a restricted test I'm not sure I expect any genuinely significant version of that before the end of the world (unrestricted Turing test passing should be sufficient unto FOOM). I'm not sure what you 'ought' to take seriously if you didn't take computers seriously in the first place. Aubrey was very specific in his prediction that I disagree with, people who forecast watershed opinion-changing events for AI are less so at least as far as I can recall.

Comment author: 14 June 2013 09:37:24AM 4 points [-]

unrestricted Turing test passing should be sufficient unto FOOM

I tend to agree, but I have to note the surface similarity with Hofstadter's disproved "No, I'm bored with chess. Let's talk about poetry." prediction.

Comment author: 11 June 2013 06:16:16PM *  2 points [-]

LW consensus is not necessarily wrong, even if Scott is right. However, making up your mind on unsettled empirical questions (which is what LW had done if Scott is right) is a dangerous practice.

I found the phrasing "he then moves in a direction that's very far from any kind of LW consensus" broadly similar to "he's not accepting the Nicene Creed, good points though he may make." Is there even a non-iffy reason to say this about an academic paper?

Comment author: 11 June 2013 07:59:22PM *  4 points [-]

I was trying to position the paper in terms of LW opinions, because my target audience were LW readers. (That's also the reason I mentioned the tangential Eliezer reference.) It's beneath my dignity to list all the different philosophical questions where my opinion is different from LW consensus, so let's just say that I used the term as a convenient reference point rather than a creed.

## [link] Scott Aaronson on free will

20 10 June 2013 11:24PM

Scott Aaronson has a new 85 page essay up, titled "The Ghost in the Quantum Turing Machine". (Abstract here.) In Section 2.11 (Singulatarianism) he explicitly mentions Eliezer as an influence. But that's just a starting point, and he then moves in a direction that's very far from any kind of LW consensus. Among other things, he suggests that a crucial qualitative difference between a person and a digital upload is that the laws of physics prohibit making perfect copies of a person. Personally, I find the arguments completely unconvincing, but Aaronson is always thought-provoking and fun to read, and this is a good excuse to read about things like (I quote the abstract) "the No-Cloning Theorem, the measurement problem, decoherence, chaos, the arrow of time, the holographic principle, Newcomb's paradox, Boltzmann brains, algorithmic information theory, and the Common Prior Assumption". This is not just a shopping list of buzzwords, these are all important components of the author's main argument. It unfortunately still seems weak to me, but the time spent reading it is not wasted at all.

Comment author: 02 February 2013 03:47:00PM 4 points [-]

An agent who only recognises finitely many utility levels doesn't have this problem. However, there's an equivalent problem for such an agent where you ask them to name a number n, and then you send them to Hell with probability 1/n and Heaven otherwise.

Comment author: 02 February 2013 05:59:12PM *  1 point [-]

If it really has only finitely many utility levels, then for a sufficiently small epsilon and some even smaller delta, it will not care whether it ends up in Hell with probability epsilon or probability delta.

View more: Next