Comment author: Vulture 23 April 2014 06:49:40PM *  14 points [-]

Pure curiousity question: What is the general status of UDT vs. TDT among yall serious FAI research people? MIRI's publications seem to exclusively refer to TDT; people here on LW seem to refer pretty much exclusively to UDT in serious discussion, at least since late 2010 or so; I've heard it reported variously that UDT is now standard because TDT is underspecified, and that UDT is just an uninteresting variant of TDT so as to hardly merit its own name. What's the deal? Has either one been fully specified/formalized? Why is there such a discrepancy between MIRI's official work and discussion here in terms of choice of theory?

Comment author: gsastry 25 April 2014 06:38:21AM 2 points [-]

Has either one been fully specified/formalized?

Here's one attempt to further formalize the different decision procedures: http://commonsenseatheism.com/wp-content/uploads/2014/04/Hintze-Problem-class-dominance-in-predictive-dilemmas.pdf (H/T linked by Luke)

Comment author: lukeprog 10 April 2014 06:07:05PM 0 points [-]
Comment author: gsastry 23 April 2014 06:33:46AM 0 points [-]
Comment author: Wei_Dai 16 March 2014 05:04:06AM 7 points [-]

What do you think are the most interesting philosophical problems within our grasp to be solved?

I'm not sure there is any. A big part of it is that metaphilosophy is essentially a complete blank, so we have no way of saying what counts as a correct solution to a philosophical problem, and hence no way of achieving high confidence that any particular philosophical problem has been solved, except maybe simple (and hence not very interesting) problems, where the solution is just intuitively obvious to everyone or nearly everyone. It's also been my experience that any time we seem to make real progress on some interesting philosophical problem, additional complications are revealed that we didn't foresee, which makes the problem seem even harder to solve than before the progress was made. I think we have to expect this trend to continue for a while yet.

If you instead ask what are some interesting philosophical problems that we can expect visible progress on in the near future, I'd cite decision theory and logical uncertainty, just based on how much new effort people are putting into them, and results from the recent past.

Do you think that solving normative ethics won't happen until a FAI? If so, why?

No I don't think that's necessarily true. It's possible that normative ethics, metaethics, and metaphilosophy are all solved before someone builds an FAI, especially if we can get significant intelligence enhancement to happen first. (Again, I think we need to solve metaethics and metaphilosophy first, otherwise how do we know that any proposed solution to normative ethics is actually correct?)

You argued previously that metaphilosophy and singularity strategies are fields with low hanging fruit. Do you have any examples of progress in metaphilosophy?

Unfortunately, not yet. BTW I'm not saying these are fields that definitely have low hanging fruit. I'm saying these are fields that could have low hanging fruit, based on how few people have worked in them.

Do you have any role models?

I do have some early role models. I recall wanting to be a real-life version of the fictional "Sandor Arbitration Intelligence at the Zoo" (from Vernor Vinge's novel A Fire Upon the Deep) who in the story is known for consistently writing the clearest and most insightful posts on the Net. And then there was Hal Finney who probably came closest to an actual real-life version of Sandor at the Zoo, and Tim May who besides inspiring me with his vision of cryptoanarchy was also a role model for doing early retirement from the tech industry and working on his own interests/causes.

Comment author: gsastry 24 March 2014 06:53:51PM 2 points [-]

Thanks. I have some followup questions :)

  1. What projects are you currently working on?/What confusing questions are you attempting to answer?
  2. Do you think that most people should be very uncertain about their values, e.g. altruism?
  3. Do you think that your views about the path to FAI are contrarian (amongst people working on FAI/AGI, e.g. you believing most of the problems are philosophical in nature)? If so, why?
  4. Where do you hang out online these days? Anywhere other than LW?

Please correct me if I've misrepresented your views.

Comment author: Wei_Dai 15 March 2014 03:30:08AM *  14 points [-]

I've been getting an increasing number of interview requests from reporters and book writers (stemming from my connection with Bitcoin). In the interest of being lazy, instead of doing more private interviews I figure I'd create an entry here and let them ask questions publicly, so I can avoid having to answer redundant questions. I'm also open to answering any other questions of LW interest here.

In preparation for this AMA, I've updated my script for retrieving and sorting all comments and posts of a given LW user, to also allow filtering by keyword or regex. So you can go to http://www.ibiblio.org/weidai/lesswrong_user.php, enter my username "Wei_Dai", then (when the page finishes loading) enter "bitcoin" in the "filter by" box to see all of my comments/posts that mention Bitcoin.

Comment author: gsastry 16 March 2014 01:21:39AM 5 points [-]
  1. What do you think are the most interesting philosophical problems within our grasp to be solved?
  2. Do you think that solving normative ethics won't happen until a FAI? If so, why?
  3. You argued previously that metaphilosophy and singularity strategies are fields with low hanging fruit. Do you have any examples of progress in metaphilosophy?
  4. Do you have any role models?