You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

lukeprog comments on Q&A with new Executive Director of Singularity Institute - Less Wrong Discussion

26 Post author: lukeprog 07 November 2011 04:58AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (177)

You are viewing a single comment's thread. Show more comments above.

Comment author: lukeprog 13 November 2011 05:19:34PM 2 points [-]

How are you going to address the perceived and actual lack of rigor associated with SIAI?

A clarifying question. By 'rigor', do you mean the kind of rigor that is required to publish in journals like Risk Analysis or Minds and Machines, or do you mean something else by 'rigor'?

Comment author: quartz 14 November 2011 09:23:34AM 7 points [-]

A clarifying question. By 'rigor', do you mean the kind of rigor that is required to publish in journals like Risk Analysis or Minds and Machines, or do you mean something else by 'rigor'?

I mean the kind of precise, mathematical analysis that would be required to publish at conferences like NIPS or in the Journal of Philosophical Logic. This entails development of technical results that are sufficiently clear and modular that other researchers can use them in their own work. In 15 years, I want to see a textbook on the mathematics of FAI that I can put on my bookshelf next to Pearl's Causality, Sipser's Introduction to the Theory of Computation and MacKay's Information Theory, Inference, and Learning Algorithms. This is not going to happen if research of sufficient quality doesn't start soon.

Comment author: lukeprog 14 November 2011 09:26:16AM 2 points [-]

In 15 years, I want to see a textbook on the mathematics of FAI that I can put on my bookshelf next to Pearl's Causality, Sipser's Introduction to the Theory of Computation and MacKay's Information Theory, Inference, and Learning Algorithms.

My day brightened imagining that!

Thanks for clarifying.

Comment author: quartz 16 November 2011 08:26:45PM 0 points [-]

Addendum: Since the people who upvoted the question were in the same position as you with respect to its interpretation, it would be good to not only address my intended meaning, but all major modes of interpretation.

Comment author: XiXiDu 13 November 2011 06:23:31PM *  3 points [-]

By 'rigor', do you mean the kind of rigor that is required to publish in journals like Risk Analysis or Minds and Machines, or do you mean something else by 'rigor'?

I can't speak for the original questioner, but take for example the latest post by Holden Karnofsky from GiveWell. I would like to see a response by the SIAI that applies the same amount of mathematical rigor to show that it actually is the rational choice from the point of view of charitable giving.

A potential donor might currently get the impression that the SIAI has written a lot of rather colloquial posts on rationality than rigorous papers on the nature of AGI, not to mention friendly AI. In contrast, GiveWell appears to concentrate on their main objective, the evaluation of charities. In doing so they are being strictly technical, an appraoch that introduces a high degree of focus by tabooing colloquial language and thereby reducing ambiguity, while allowing others to review their work.

Some of the currently available papers might, in a less favorably academic context, be viewed as some amount of handwaving mixed with speculations.