Wei_Dai comments on The Importance of Self-Doubt - Less Wrong

23 Post author: multifoliaterose 19 August 2010 10:47PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (726)

You are viewing a single comment's thread. Show more comments above.

Comment author: Wei_Dai 03 September 2010 03:50:31PM *  3 points [-]

Right, but the historical precedent for an amateur scientist even being at all involved in a substantial scientific breakthrough over the past 50 years is very weak.

What are we supposed to infer from that? That if you add an amateur scientist to a group of PhDs, that would substantially decrease their chance of making a breakthrough?

The impressions that I've gotten from my private correspondence with Eliezer and from his comments have given me a very strong impression that I would find him too difficult to work with for me to be able to do productive FAI research with him.

SIAI held a 3-day decision theory workshop in March that I attended along with Stuart Armstrong and Gary Drescher as outside guests. I feel pretty safe in saying that none of us found Eliezer particularly difficult to work with. I wonder if perhaps you're generalizing from one example here.

I think that it would be worthwhile for somebody to do FAI research in line with Vladimir Nesov's remarks here and here.

Do you also think it would be worthwhile for somebody to try to build an organization to do FAI research? If so, who do you think should be doing that, if not Eliezer and his supporters? Or is your position more like cousin_it's, namely that FAI research should just be done by individuals on their free time for now?

Comment author: multifoliaterose 03 September 2010 04:15:06PM 0 points [-]

What are we supposed to infer from that? That if you add an amateur scientist to a group of PhDs, that would substantially decrease their chance of making a breakthrough?

No, certainly not. I just don't see much evidence that Eliezer is presently adding value to Friendly AI research. I think he could be doing more to reduce existential risk if he were operating under different assumptions.

SIAI held a 3-day decision theory workshop in March that I attended along with Stuart Armstrong and Gary Drescher as outside guests. I feel pretty safe in saying that none of us found Eliezer particularly difficult to work with. I wonder if perhaps you're generalizing from one example here.

Of course you could be right here, but the situation is symmetric, the same could be the case for you, Stuart Armstrong and Gary Drescher. Keep in mind that there's a strong selection effect here - if you're spending time with Eliezer you're disproportionately likely to be well suited to working with Eliezer, and people who have difficulty working with Eliezer are disproportionately unlikely be posting on Less Wrong or meeting with Eliezer.

My intuition is that there are a lot of good potential FAI researchers who would not feel comfortable working with Eliezer given his current disposition, but I may be wrong.

Do you also think it would be worthwhile for somebody to try to build an organization to do FAI research? If so, who do you think should be doing that, if not Eliezer and his supporters? Or is your position more like cousin_it's, namely that FAI research should just be done by individuals on their free time for now?

Quite possibly it's a good thing for Eliezer and his supporters to be building an organization to do FAI research. On the other hand maybe cousin_it's position is right. I have a fair amount of uncertainty on this point.

The claim that I'm making is quite narrow: that it would be good for the cause of existential risk reduction if Eliezer seriously considered the possibility that he's greatly overestimated his chances of building a Friendly AI.

I'm not saying that it's a bad thing to have an organization like SIAI. I'm not saying that Eliezer doesn't have a valuable role to serve within SIAI. I'm reminded of Robin Hanson's Against Disclaimers though I don't feel comfortable with his condescending tone and am not thinking of you in that light :-).