I don't live in Australia.

Wow. Next Tuesday and in my hometown! Nice.

Sigh... I would consider flying down from Sydney to go to it, but sadly I'm in a show that whole week and have to miss out entirely. Ah well. Hopefully they'll have the audio online, but I would have loved to mingle with people who share my worldview.

Ping Arthur breitman fb or linked in. He is part of NYC lw meetup, and a quant at goldman.

I am, and I am planning to leave it to get a higher more average pay. From my viewpoint, it is terribly overrated and undervalued.

If N is at least 18 it’s hard to think of a rational criteria under which believing you are 1 in 10^N is delusional whereas thinking you are 1 in 10^(N-12) is not.

Really? How about "when you are, in fact, 1/10^(N-12) and have good reason to believe it"? Throwing in a large N doesn't change the fact that 10^N is still 1,000,000,000,000 times larger than 10^(N-12) and nor does it mean we could not draw conclusions about belief (2).

(Not commenting on Eliezer here, just suggesting the argument is not all that persuasive to me.)

Can you provide a cite for the notion that Eliezer believes (2)? Since he's not likely to build the world's first FAI in his garage all by himself, without incorporating the work of any of other thousands of people working on FAI and FAI's necessary component technologies, I think it would be a bit delusional of him to beleive (2) as stated. Which is not to suggest that his work is not important, or even among the most significant work done in the history of humankind (even if he fails, others can build on that and find the way that works). But that's different than the idea that he, alone, is The Most Significant Human Who Will Ever Live. I don't get the impression that he's that cocky.

2 is ambiguous. Getting to the stars requires a number of things to go right. Eliezer serves relatively little use in preventing a major nuclear exchange in the next 10 years, or bad nanotech , or garage made bio weapons, or even UFAI development.

FAI is just the final thing that needs to go right, everything else needs to go mostly right until then.

I think I don't understand (1) and its implications. How the fact that in most of the branches we are going extinct implies that we are the most important couple of generations (this is how I interpret the trillion)? Our importance lies in our decisions. These decisions influence the number of branches in which people die out. If we take (1) as given, it means we weren't successful in mitigating the existential risk, leaving no place to excercise our decisions and thus importance.

It's not about the numbers, and it's not about Eliezer in particular. Think of it this way:

Clearly, the development of interstellar travel (if we successfully accomplish this) will be one of the most important events in the history of the universe.

If I believe our civilization has a chance of achieving this, then in a sense that makes me, as a member of said civilization, important. This is a rational conclusion.

If I believe I'm going to build a starship in my garage, that makes me delusional. The problem isn't the odds against me being the one person who does this. The problem is that nobody is going to do this, because building a starship in your garage is simply impossible; it's just too hard a job to be done that way.

It seems that I am not one in a million. Pity.

Once there is recursively improving AI, the human race is irrelevant; anyone planning to continue living under those conditions either has not thought things through, would be equally happy living in a permanent simulation, or as a wirehead.

This does not follow.

I don’t care if AI is Friendly or not. [...] I am mainly interested in that whatever AI we create does not paperclip the universe

You contradict yourself here. A Friendly AI is an intelligence which attempts to improve the well-being of humanity. A paperclip maximiser is an intelligence which does not, as it cares about something different and unrelated. Any sufficiently advanced AI is either one or the other or somewhere in between.

By "sufficiently advanced", I mean an AI which is intelligent enough to consider the future of humanity and attempt to influence it.

anyone planning to continue living under those conditions either has not thought things through, would be equally happy living in a permanent simulation, or as a wirehead.

Your existence may not be relevant to the rest of the universe after friendly AI, but that doesn't mean that you would be a wirehead. I want to live a life of genuine challenges, but I really wish that it didn't have to be in a world of genuine suffering.

It is very difficult to distinguish rationalisations of the discomfort of change, with actual consequences. If this belief that hypertext leads to a less sophisticated understanding than reading a book, what behaviour would change that could be measured?

I haven't read Nicholas Carr, but I've seen summaries of some of the studies used to claim that book reading results in more comprehension than hypertext reading. All the ones I saw are bogus. They all use, for the hypertext reading, a linear extract from a book, broken up into sections separated by links. Sometimes the links are placed in somewhat arbitrary places. Of course a linear text can be read more easily linearly.

I believe hypertext reading is deeper, and that this is obvious, almost true by definition. Non-hypertext reading is exactly 1 layer deep. Hypertext lets the reader go deeper. Literally. You can zoom in on any topic.

A more fair test would be to give students a topic to study, with the same material, but some given books, and some given the book material organized and indexed in a competent way as hypertext.

Wide and deep reading, such that you make the information presented yours, gives you more background knowledge that helps you find your own connections.

Hypertext reading lets you find your own connections, and lets you find background knowledge that would otherwise simply be edited out of a book.

This sounds interesting and relevant. Here's my input: I read this back in 2008 and I am summarising it from memory, so I may make a few factual errors. But I read that one of the problem facing large Internet companies like Google is the size of their server farms, which need cooling, power, space, etc. Optimising the algorithms used can help enormously. A particular program was responsible for allocating system resources so that the systems which were operating were operating at near full capacity, and the rest could be powered down to save energy. Unfortunately, this program was executed many times a second, to the point where the savings it created were much less than the power it used. The fix was simply to execute it less often. Running the program took about the same amount of time no mater how many inefficiencies it detected, so it was not worth checking the entire system for new problems if you only expected to find one or two.

My point: To reduce resources spent on decision-making, make bigger decisions but make them less often. Small problems can be ignored fairly safely, and they may be rendered irrelevant once you solve the big ones.

I was having similar thoughts the other day while watching a reality TV show where designers competed for a job from Philippe Starck. Some of them spent ages trying to think of a suitable project, and then didn't have enough time to complete it; some of them launched into the first plan they had and it turned out rubbish. Clearly they needed some meta-planning. But how much? Well, they'll need to do some meta-meta planning...

I'd be happy to give your post a read through.

ETA: The buck stops immediately, of course.

Upvoted for importance of subject - looking forward to the post. Have you read up on Information Foraging?

He did give a permission to restore the posts (I didn't ask about comments), when I contacted him originally. There remains the issue of someone being technically able to restore these posts.

That's very interesting, but I'm not sure I trust the article's statistics, and I don't have access to the full text. Could someone take a closer look and confirm that there are no shennanigans going on?

The study looks at people over 55 years of age. It is possible that there is some sort of selection effect going on -- maybe decades of heavy drinking will weed out all but the most alcohol-resistant individuals, so that those who are still drinking heavily at 55-60 without ever having been harmed by it are mostly immune to the doses they're taking. From what I see, the study controls for past "problem drinking" (which they don't define precisely), but not for people who drank heavily without developing a drinking problem, but couldn't handle it any more after some point and decided themselves to cut back.

Also, it should be noted that papers of this sort use pretty conservative definitions of "heavy drinking." In this paper, it's defined as more than 42 grams of alcohol per day, which amounts to about a liter of beer or three small glasses of wine. While this level of drinking would surely be risky for people who are exceptionally alcohol-intolerant or prone to alcoholism, lots of people can handle it without any problems at all. It would be interesting to see a similar study that would make a finer distinction between different levels of "heavy" drinking.

The discussion of the same paper on Overcoming Bias has reminded me of another striking correlation I read about recently:
http://www.marginalrevolution.com/marginalrevolution/2010/07/beer-makes-bud-wiser.html

It seems that for whatever reason, abstinence does correlate with lower performance on at least some tests of mental ability. The question is whether the controls in the study cover all the variables through which these lower abilities might have manifested themselves in practice; to me it seems quite plausible that the answer could be no.

These are fine conclusions to live by, as long as moderate drinking doesn't lead you to heavy drinking, cirrhosis and the grave. Come visit Russia to take a look.

If there's just one topic that's banned, then no. If it's increased to 2 topics - and "No riddle theory" is one I hadn't heard before - then maybe. Moderation and deletion is very rare here.

I would like moderation or deletion to include sending an email to the affected person - but this relies on the user giving a good email address at registration.

I think such discussion wouldn't necessarily warrant its own top-level post, but I think it would fit well in a new Meta thread. I have been meaning to post such a thread for a while, since there are also a couple of meta topics I would like to discuss, but I haven't gotten around to it.

Do people think that a discussion forum on the moderation and deletion policies would be beneficial?

I don't. Possible downsides are flame wars among people who support different types of moderation policies (and there are bound to be some - self-styled rebels who pride themselves in challenging the status quo and going against groupthink are not rare on the net), and I don't see any possible upsides. Having a Benevolent Dictator For Life works quite well.

See this on Meatball Wiki, that has quite a few pages on organization of Online Communities.

Do people think that a discussion forum on the moderation and deletion policies would be beneficial?

I would like to see a top-level post on moderation policy. But I would like for it to be written by someone with moderation authority. If there are special rules for discussing moderation, they can be spelled out in the post and commenters can abide by them.

As a newcomer here, I am completely mystified by the dark hints of a forbidden topic. Every hypothesis I can come up with as to why a topic might be forbidden founders when I try to reconcile with the fact that the people doing the forbidding are not stupid.

Self-censorship to protect our own mental health? Stupid. Secrecy as a counter-intelligence measure, to safeguard the fact that we possess some counter-measure capability? Stupid. Secrecy simply because being a member of a secret society is cool? Stupid, but perhaps not stupid enough to be ruled out. On the other hand, I am sure that I haven't thought of every possible explanation.

It strikes me as perfectly reasonable if certain topics are forbidden because discussion of such topics has historically been unproductive, has led to flame wars, etc. I have been wandering around the internet long enough to understand and even appreciate somewhat arbitrary, publicly announced moderation policies. But arbitrary and secret policies are a prescription for resentment and for time wasted discussing moderation policies.

Edit: typo correction - insert missing words

Your assessment is along the right lines, though if anything a little optimistic; uploading is an enormously difficult engineering challenge, but at least we can see in principle how it could be done, and recognize when we are making progress, whereas with AI we don't yet even have a consensus on what constitutes progress.

I'm personally working on AI because I think that's where my talents can be best used, and I think it can deliver useful results well short of human equivalence, but if you figure you'd rather work on uploading, that's certainly a reasonable choice.

As for what uploads will do if and when they come to exist, well, there's going to be plenty of time to figure that out, because the first few of them are going to spend the first few years having conversations like,

"Uh... a hatstand?"

"Sorry Mr. Jones, that's actually a picture of your wife. I think we need to revert yesterday's bug fixes to your visual cortex."

But e.g. The Planck Dive is a good story set in a world where that technology is mature enough to be taken for granted.

Emulating an entire brain, and finding out how the higher intelligence parts work and adapting them for practical purposes, are two entirely different achievements. Even if you could upload a brain onto your computer and let it run, it would be absurdly slow, however, simulating some kind of new optimization process we find from it might be plausible.

And either way, don't expect a singularity anytime soon with that. Scientists believe it took thousands of years after modern intelligence emerged for us to learn symbolic thought. Then thousands more before we discovered the scientific method. It's only now we are finally discovering rational thinking. Maybe an AI could start where we left off, or maybe it would take years before it could even get to the level to be able to do that, and then years more before it could make the jump to the next major improvement, assuming there even is one.

I'm not arguing against AI here at all. I believe a singularity will probably happen and soon, but Emulation is definitely not the way to go. Humans have way to many flaws we don't even know would be possible to fix, even if we knew what the problem was in the first place.

What is the ultimate goal in the first place? To do something along the lines of replicating brains of some of the most intelligent people and forcing them to work on improving humanity/developing AI? Has anyone considered there is a far more realistic way of doing this through cloning, eugenics, education research, etc. Of course no one would do it because it is amoral, but then again, what is the difference between the two?

When I tried making sense of this topic in the context of the controversies over IQ heritability, the best reference I found was this old paper:

Brian Mackenzie, Fallacious use of regression effects in the I.Q. controversy, Australian Psychologist 15(3):369-384, 1980

Unfortunately, the paper failed to achieve any significant impact, probably because it was published in a low-key journal long before Google, and it's now languishing in complete obscurity. I considered contacting the author to ask if it could be put for open access online -- it would be definitely worth it -- but I was unable to find any contact information; it seems like he retired long ago.

There is also another paper with a pretty good exposition of this problem, which seems to be a minor classic, and is still cited occasionally:

Lita Furby, Interpreting regression toward the mean in developmental research, Developmental Psychology, 8(2):172-179, 1973

Wow. I thought I understood regression to the mean already, but the "correlation between X and Y-X" is so much simpler and clearer than any explanation I could give.

This community already hopefully accepts that one can learn from knowing other people's opinions without knowing their arguments; Aumann's agreement theorem, and so forth.

I don't think Aumann's agreement theorem has anything to do with taking people's opinions as evidence. Aumann's agreement theorem is about agents turning out to have been agreeing all along, given certain conditions, not about how to come to an agreement, or worse how to enforce agreement by responding to others' beliefs.

More generally (as in, not about this particular comment), the mentions of this theorem on LW seem to have degenerated into applause lights for "boo disagreement", having nothing to do with the theorem itself. It's easier to use the associated label, even if such usage would be incorrect, but one should resist the temptation.

I'm not sure about having a centralised group doing this but I did experiment with making a tool that could help infer consequences from beliefs. Imagine something a little like this but with chains of philosophical statements that have degrees of confidence. Users would assign confidence to axioms and construct trees of argument using them. The system would automatically determine confidences of conclusions. It could even exist as a competitive game with a community determining confidence of axioms. It could also be used to rapidly determine differences in opinion i.e. infer the main inferred points of contention based on different axiom weightings. If anyone knows of anything similar or has suggestions for such a system I'd love to hear them. Including any reasons why it might fail. Because I think it's an interesting solution to the 'how to efficiently debate reasonably'.

Wow, a debate where the most reasonable-sounding person is a sysop of Conservapedia. :)

I've long had the idea of writing a sequence on aesthetics; I'm not sure if and when I'll ever get around to it, however. (I have a fairly large backlog of post ideas that have yet to be realized.)

I used to be a professional games programmer and designer and I'm very interested in fun. There are a couple of good books on the subject: A theory of fun and Rules of play. As a designer I spent many months analyzing sales figures for both computer games and other conventional toys. The patterns within them are quite interesting: for example child's toys pass from amorphous learning tools (bright objects and blobby humanoids), through mimicking parents (accurate baby dolls), to mimicking older children (sexualised dolls and makeup). My ultimate conclusions were that fun takes many forms whose source can be ultimately reduced to what motivates us. In effect, fun things are mental hacks of our intrinsic motivations. I gave a couple of talks on my take on what these motivations are. I'd be happy to repeat this material here (or upload and link to the videos if people prefer).

I don't recall ever doing that.

Do you leave the votes stand because you remember/re-invent your original reason for upvoting, or because something along the lines of "well, I must've had a good reason at the time"?

No social structure can be permanent without the biological level being fixed. And Bingo! Fukuyama being a smart man, understood this and his next book was "Our posthuman future", which urged the extreme social control of biological manipulation, in particular, ceasing research.

Really? I would have arrived at the opposite conclusion. No social structure can be permanent without the biological level being fixed, therefore we should do more research into biological alteration in order to stabilize our biology should it become unstable.

For instance, pre-implantation genetic diagnosis would enable us to almost eradicate most genetic diseases, thus maintaining our biological quality. I'm not saying it doesn't have corresponding problems, just that an attitude of "we should cease research in this field because we might find something dangerous" is overreacting.

No social structure can be permanent without the biological level being fixed.

This seems incorrect, as it's not hard to imagine a social structure supporting a wide variety of different biological/non-biological intelligences, as long as they were reasonably close to each other in morality-space. There's plenty of things at the level of biology that have no impact on morality that we'd certainly like to change.

But now I sense an even more disturbing definition: rational as opposed to empirical.

I don't think that's how most people here understand "rationalism".

But now I sense an even more disturbing definition: rational as opposed to empirical. As I use scientific evidence as the most important arbiter of what I believe, I would find the anti-empirical idea of 'rational' a big mistake.

Indeed. It is heretic in the extreme! Burn them!

In a certain sense rationality is using evidence efficiently. Perhaps overemphasis on that type of rationality tempts one to be sparing with evidence - after all if you use less evidence to reach your conclusion you used whatever evidence you did use more efficiently! But not using evidence doesn't mean there is more evidence left afterwards, not using free or very cheap evidence is wasteful, so proper rationality, even in that sense, means using all easily available evidence when practical.

But now I sense an even more disturbing definition: rational as opposed to empirical.

Ummmmmmmm.... no.

The word "rational" is used here on LW in essentially its literal definition (which is not quite the same as its colloquial everyday meaning).... if anything it is perhaps used by some to mean "bayesian"... but bayesianism is all about updating on (empirical) evidence.

There is at least one post about that - though I don't entirely approve of it.

Occam's razor is not exactly empirical. Evidence is involved - but it does let you choose between two theories both of which are compatible with the evidence without doing further observations. It is not empirical - in that sense.

Here is our definition of rationality. See also the "unnamed virtue".

The philosophical tradition of 'Rationalism' (opposed to 'Empiricism') is not relevant to the meaning here. Though there is some relationship between it and "Traditional Rationality" which is referenced sometimes.

The would be more likely to work if you completely took out the 'for existential risk' part. Find a way to cooperate with people effectively "to make money". No need to get religion all muddled up in it.

Is there a reasonable way of applying probability to analogue inference problems?

Your examples, certainly show a grasp of the problem. The solution is first sketched in Chapter 4.6 of Jaynes

Is, in effect, the selection of the underlying model the real challenge of rational decision making, not the inference rules?

Definitely. Jaynes finishes deriving the inference rules in Chapter 2 and illustrates how to use them in Chapter 3. The remainder of the book deals with "the real challenge". In particular Chapters 6, 7, 12, 19, and especially 20. In effect, you use Bayesian inference and/or Wald decision theory to choose between underlying models pretty much as you might have used them to choose between simple hypotheses. But there are subtleties, ... to put things mildly. But then classical statistics has its subtleties too.

Yes, I know that schools will never assign eight-ball for homework. But maybe there is some kid-friendly way to teach the same lesson.

Make them play some kind of simplified RPG until they realise the only achievement is how much time they put into doing mindless repetitive tasks.

But I've noticed in many people (myself included) a definite tendency to overvalue intelligence relative to practice.

I'm not sure I agree with that. In what areas do you see overvalue of intelligence relative to practice and why do you think there really is overvalue in those areas?

I've noticed for example that people's abilities to make good comments on LW do not seem to improve much with practice and feedback from votes (beyond maybe the first few weeks or so). Does this view represent an overvalue of intelligence?

I don't mean to say that talent doesn't matter: things such as intelligence matter more than Adams gives them credit for

I think the relative contribution of intelligence vs. practice varies substantially depending on the nature of the particular task. A key problem is to identify tasks as intelligence-dominated (the smart guy always wins) vs. practice-dominated (the experienced guy always wins).

As a first observation about this problem, notice that clearly definable or objective tasks (chess, pool, basketball) tend to be practice-dominated, whereas more ambiguous tasks (leadership, writing, rationality) tend to be intelligence-dominated.

people in this community are unusually prone to feeling that they're stupid if they do badly at something

I suspect this is a result of the tacit assumption that "if you're not smart enough, you don't belong at LW". If most members are anything like me, this combined with the fact that they're probably used to being "the smart one" makes it extremely intimidating to post anything, and extremely de-motivational if they make a mistake.

In the interests of spreading the idea that it's ok if other people are smarter than you, I'll say that I'm quite certainly one of the less intelligent members of this community.

I've noticed in many people (myself included) a definite tendency to overvalue intelligence relative to practice.

Practice and expertise tend to be domain-specific - Scott isn't any better at darts or chess after playing all that pool. Even learning things like metacognition tend not to apply outside of the specific domain you've learned it in. Intelligence is one of the only things that gives you a general problem solving/task completion ability.

It seems to me, based on purely anecdotal experience, that people in this community are unusually prone to feeling that they're stupid if they do badly at something.

This is certainly true of me, but I try to make sure that the positive feeling of having identified the mistakes and improved outweighs the negative feeling of having needed the improvement. Tsuyoku Naritai!

They did? I've been wishing for something like that myself. I'd also like another page that collects just my high karma comments. Extremely useful feedback!

Should I just use 'indignation' and then define what I mean in the first few sentences?

That could work well when backed up by with the description of just what you will be using the term to mean.

I will be interested to read your post - from your brief introduction here I think I have had similar observations about emotions that interfere with thought, independent of raw overwhelm from primitives like anger.

The words righteous indignation in combination are sufficiently well-recognized as to have their own wikipedia page. The page also says that righteous indignation has overtones of religiosity, which seems like a reason not to use it in your sense . It also says that it is akin to a "sense of injustice," but at least for me, that phrase doesn't have as much resonance.

Edited to add this possibly relevant/interesting link I came across, where David Brin describes self-righteous indignation as addictive.