lukeprog comments on Singularity Institute Executive Director Q&A #2 - Less Wrong

20 Post author: lukeprog 06 January 2012 03:40AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (39)
Tags: meta

Comments (39)

You are viewing a single comment's thread. Show more comments above.

Comment author: lukeprog 06 January 2012 03:50:26PM *  21 points [-]

Fair enough. I'll share a few examples of progress, though these won't be surprising to people who are on every mailing list, read every LW post, or are in the Bay Area and have regular conversations with us.

  • much progress on the strategic landscape, e.g. differential technological development analyses, which you'll see in the forthcoming Anna/Luke chapter and in Nick's forthcoming monograph, and which you've already seen in several papers and talks over the past couple years (most of them involving Carl).
  • progress on decision theory, largely via the decision theory workshop mailing list, in particular on UDT
  • progress in outlining the sub-problems of singularity research, which I've started to write up here.
  • progress on the value-loading problem, explained here and in a forthcoming paper by Dewey.
  • progress on the reflectivity problem in the sense of identifying lots of potential solutions that probably won't work. :)
  • progress on the preference extraction problem via incorporating the latest from decision neuroscience

Still, I'd say more of our work has been focused on movement-building than on cutting-edge research, because we think the most immediate concern is not on cutting-edge research but on building a larger community of support, funding, and researchers to work on these problems. Three researchers can have more of an impact if they create a platform by which 20 researchers can work on the problem than if they merely do research by themselves.

Comment author: [deleted] 06 January 2012 05:39:15PM 5 points [-]

Thank you, this is exactly the kind of answer I was hoping for.

Comment author: steven0461 07 January 2012 09:35:12PM 3 points [-]

Is the value-loading or value-learning problem the same thing as the problem of moral uncertainty? If no, what am I missing; if yes, why are the official solution candidates different?

Comment author: Bugmaster 06 January 2012 09:15:00PM 3 points [-]

Thanks, this is quite informative, especially your closing paragraph:

Still, I'd say more of our work has been focused on movement-building than on cutting-edge research, because we think the most immediate concern is not on cutting-edge research but on building a larger community of support, funding, and researchers to work on these problems.

This makes sense to me; have you considered incorporating this paragraph into your core mission statement ? Also, what are your thresholds for deciding when to transition from (primarily) community-building to (primarily) doing research ?

Also, you mentioned (in your main post) that the SIAI has quite a few papers in the works, awaiting publication; and apparently there are even a few books waiting for publishers. Would it not be more efficient to post the articles and books in question on Less Wrong, or upload them to Pirate Bay, or something to that extent -- at least, while you wait for the meat-space publishers to get their act together ? Sorry if this is a naive question; I know very little about the publishing world.

Comment author: lukeprog 06 January 2012 09:36:04PM 1 point [-]

what are your thresholds for deciding when to transition from (primarily) community-building to (primarily) doing research ?

We're not precisely sure. It's also a matter of funding. Researchers who can publish "platform research" for academic outreach, problem space clarification, and community building are less expensive than researchers who can solve decision theory, safe AI architectures, etc.

Would it not be more efficient to post the articles and books in question on Less Wrong, or upload them to Pirate Bay, or something to that extent -- at least, while you wait for the meat-space publishers to get their act together ?

Like many academics, we generally do publish early drafts of forthcoming articles long before the final version is written and published. Examples: 1, 2, 3, 4.

Comment author: amcknight 07 January 2012 12:34:56AM 2 points [-]

progress on the preference extraction problem via incorporating the latest from decision neuroscience

I'd love to hear more about what areas you're looking into within decision neuroscience.

For those who are also interested and somehow missed these:
Crash Course in Neuroscience of Motivation
and these two neuroeconomics book reviews.

Comment author: lukeprog 07 January 2012 12:54:50AM 1 point [-]

An example: The subject matter of the second chapter of this book (the three competing systems of motivation) looks to have some implications for the value extraction problem. This is the kind of information about how our preferences work that I imagine we'll use to extrapolate our preferences — or that an AI would use to do the extrapolation for us.

Powered by Reddit Powered by Reddit