komponisto comments on Common sense as a prior - Less Wrong

33 Post author: Nick_Beckstead 11 August 2013 06:18PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (212)

You are viewing a single comment's thread. Show more comments above.

Comment author: JonahSinick 11 August 2013 06:29:13AM *  1 point [-]

Thanks for the thoughtful response. I agree that there are many instances in which it’s possible to rationally come to confident conclusions that differ from those of subject matter experts. I realize that my earlier comment was elliptical, and will try to clarify. The relevant points to my mind are:

The extraordinary intellectual caliber of the best physicists

Though difficult to formalize, I think that there's a meaningful sense in which one can make statements of the general form "person A has intellectual caliber n times that of person B." Of course, this is domain specific to some degree, but I think that the concept hangs together somewhat even across domains.

One operationalization of this is "if person B reaches a correct conclusion on a given subject, person A could reach it n times as fast." Another is "it would take n copies of person B to do person A's work." These things are hard to estimate, but one can become better calibrated by using the rule "if person A has intellectual caliber n times that of person B and person B has intellectual caliber m times that of person C, then person A has intellectual caliber n*m times that of person C."

In almost all domains, I think that the highest intellectual caliber people have no more than 5x my intellectual caliber. Physics is different. From what I’ve heard, the distribution of talent in physics is similar to that of math. The best mathematicians are 100x+ my intellectual caliber. I had a particularly striking illustrative experience with Don Zagier, who pinpointed a crucial weakness in an analogy that I had been exploring for 6 months (and which I had run by a number of other mathematicians) in a mere ~15 minutes. I would not be surprised if he himself were to have an analogous experience with the historical greats.

When someone is < 5x one’s intellectual caliber, an argument of the type “this person may be smarter than me, but I’ve focused a lot more on having accurate views, so I trust my judgment over him or her” seems reasonable. But when one gets to people who are 100x+ one’s intellectual caliber, the argument becomes much weaker. Model uncertainty starts to play a major role. It could be that people who are that much more powerful easily come to the correct conclusion on a given question without even needing to put conscious effort into having accurate beliefs.

The intrinsic interest of the question of interpretation of quantum mechanics

The question of what quantum mechanics means has been considered one of the universe’s great mysteries. As such, people interested in physics have been highly motivated to understand it. So I think that the question is privileged relative to other questions that physicists would have opinions on — it’s not an arbitrary question outside of the domain of their research accomplishments.

Solicitation of arguments from those with opposing views

In the Muslim theology example, you spend 40 hours engaging with the Muslim philosophers. This seems disanalogous to the present case, in that as far as I know, Eliezer’s quantum mechanics sequence hasn’t been vetted by any leading physicists who disagree with the many world’s interpretation of quantum mechanics. I also don’t know of any public record of ~40 hours of back and forth analogous to the one that you describe. I know that Eliezer might cite an example in his QM sequence, and will take a look.

Comment author: komponisto 13 August 2013 03:11:18AM *  9 points [-]

Let me first say that I find this to be an extremely interesting discussion.

In almost all domains, I think that the highest intellectual caliber people have no more than 5x my intellectual caliber. Physics is different. From what I’ve heard, the distribution of talent in physics is similar to that of math. The best mathematicians are 100x+ my intellectual caliber.

I think there is a social norm in mathematics and physics that requires people to say this, but I have serious doubts about whether it is true. Anyone 100x+ your intellectual caliber should be having much, much more impact on the world (to say nothing of mathematics itself) than any of the best mathematicians seem to be having. At the very least, if there really are people of that cognitive level running around, then the rest of the world is doing an absolutely terrible job of extracting information and value from them, and they themselves must not care too much about this fact.

More plausible to me is the hypothesis that the best mathematicians are within the same 5x limit as everyone else, and that you overestimate the difficulty of performing at their level due to cultural factors which discourage systematic study of how to imitate them.

Try this thought experiment: suppose you were a graduate student in mathematics, and went to your advisor and said: "I'd like to solve [Famous Problem X], and to start, I'm going to spend two years closely examining the work of Newton, Gauss, and Wiles, and their contemporaries, to try to discern at a higher level of generality what the cognitive stumbling blocks to solving previous problems were, and how they overcame them, and distill these meta-level insights into a meta-level technique of my own which I'll then apply to [Famous Problem X]." What do you think the reaction would be? How many times do you think such a thing has ever been proposed, let alone attempted, by a serious student or (even) senior researcher?

Comment author: JonahSinick 13 August 2013 04:16:24AM 4 points [-]

Nice to hear from you :-)

At the very least, if there really are people of that cognitive level running around, then the rest of the world is doing an absolutely terrible job of extracting information and value from them, and they themselves must not care too much about this fact.

Yes, I think that this is what the situation is.

I'll also say that I think that there are very few such people — maybe on the order of 10 who are alive today. With such a small absolute number, I don't think that their observed impact on math is a lot lower than what one would expect a priori, and the prior in favor them having had a huge impact in society isn't that strong.

More plausible to me is the hypothesis that the best mathematicians are within the same 5x limit as everyone else, and that you overestimate the difficulty of performing at their level due to cultural factors which discourage systematic study of how to imitate them.

"The best mathematicians are 100+x higher in intellectual caliber than I am" and "the difference is in large part due to cultural factors which discourage systematic study of how to imitate them" aren't mutually exclusive. I'm sympathetic to your position.

What do you think the reaction would be?

To change the subject :-)

How many times do you think such a thing has ever been proposed, let alone attempted, by a serious student or (even) senior researcher?

Basically never.

Comment author: [deleted] 13 August 2013 10:05:29AM 0 points [-]

I'll also say that I think that there are very few such people — maybe on the order of 10 who are alive today.

Not to mention that some of them might be working on Wall Street or something, and not have worked on unsolved problems in mathematics in decades.

Comment author: Eliezer_Yudkowsky 13 August 2013 04:22:04AM 8 points [-]

Try this thought experiment: suppose you were a graduate student in mathematics, and went to your advisor and said: "I'd like to solve [Famous Problem X], and to start, I'm going to spend two years closely examining the work of Newton, Gauss, and Wiles, and their contemporaries, to try to discern at a higher level of generality what the cognitive stumbling blocks to solving previous problems were, and how they overcame them, and distill these meta-level insights into a meta-level technique of my own which I'll then apply to [Famous Problem X]."

This is a terrible idea unless they're spending half their time pushing their limits on object-level math problems. I just don't think it works to try to do a meta phase before an object phase unless the process is very, very well-understood and tested already.

Comment author: komponisto 13 August 2013 10:22:48AM 2 points [-]

I'm sure that's exactly what the advisor would say (if they bother to give a reasoned reply at all), with the result that nobody ever tries this.

(I'll also note that it's somewhat odd to hear this response from someone whose entire mission in life is essentially to go meta on all of humanity's problems...)

But let me address the point, so as not to be logically rude. The person would be pushing their limits on object-level math problems in the course of "examining the work of Newton, Gauss, and Wiles", in order to understand said work; otherwise, it can hardly be said to constitute a meaningful examination. I also think it's important not to confuse meta-ness with (nontechnical) "outside views"; indeed I suspect that a lot of the thought processes of mathematical "geniuses" consist of abstracting over classes of technical concepts that aren't ordinarily abstracted over, and thus if expressed explicitly (which the geniuses may lack the patience to do) would simply look like another form of mathematics. (Others of their processes, I speculate, consist in obsessive exercising of visual/dynamic mental models of various abstractions.)

Switching back to logical rudeness, I'm not sure the meta-ness is your true rejection; I suspect what you may be really worried about is making sure there are tight feedback loops to which one's reasoning can be subjected.

Comment author: Eliezer_Yudkowsky 13 August 2013 08:05:40PM 8 points [-]

(I'll also note that it's somewhat odd to hear this response from someone whose entire mission in life is essentially to go meta on all of humanity's problems...)

That's not the kind of meta I mean. The dangerous form of meta is when you spend several years preparing to do X, supposedly becoming better at doing X, but not actually doing X, and then try to do X. E.g. college. Trying to improve at doing X while doing X is much, much wiser. I would similarly advise Effective Altruists who are not literally broke to be donating $10 every three months to something while they are trying to increase their incomes and invest in human capital; furthermore, they should not donate to the same thing two seasons in a row, so that they are also practicing the skill of repeatedly assessing which charity is most important.

"Meta" for these purposes is any daily activity which is unlike the daily activity you intend to do 'later'.

Tight feedback loops are good, but not always available. This is a separate consideration from doing meta while doing object.

The activity of understanding someone else's proofs may be unlike the activity of producing your own new math from scratch; this would be the problem.

Comment author: somervta 15 August 2013 09:19:54AM 3 points [-]

I would similarly advise Effective Altruists who are not literally broke to be donating $10 every three months to something while they are trying to increase their incomes and invest in human capital; furthermore, they should not donate to the same thing two seasons in a row, so that they are also practicing the skill of repeatedly assessing which charity is most important.

This is excellent advice. I have put a note in my calendar thee months hence to reevaluate my small monthly donation.

Comment author: [deleted] 13 August 2013 10:00:59AM -1 points [-]

Anyone 100x+ your intellectual caliber should be having much, much more impact on the world (to say nothing of mathematics itself) than any of the best mathematicians seem to be having.

How do you know how little intellectual caliber JonahSinick has?

looks at JonahSinick's profile

follows link to his website

skims the “About me” page

Yes, you have a point.