NihilCredo comments on Let's make a deal - Less Wrong

-18 Post author: Mitchell_Porter 23 September 2010 12:59AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (54)

You are viewing a single comment's thread. Show more comments above.

Comment author: Will_Newsome 23 September 2010 03:21:19AM *  21 points [-]

I think it's really stupid that people have to work stupid jobs to do actually valuable things. I also feel somewhat bad about making a critique, as I don't have money to fund you even if you satisfactorily responded to it. Nonetheless, I feel someone should respond in a little more detail, so as to hopefully set a precedent as to the standards that need to be met before requests for funding are made.

If such theory is important to Friendliness, Eliezer or Marcello should be alerted. If your approach is important to Friendliness, Eliezer or Marcello should convince SIAI to fund you. If Eliezer or Marcello don't deem your approach worth funding, then to many people that is pretty strong evidence against the merit of your approach. To convince those people you would have to show either where Eliezer or Marcello are wrong in their critique, or where Eliezer or Marcello are likely to go wrong in general when considering potential approaches to Friendliness. Have you tried talking to Eliezer or Marcello? If not, can you provide evidence that they are wrong in deeming it not worth talking about your approach to Friendliness?

For those that are interested in Friendliness but do not think Marcello or Eliezer are likely to notice correct approaches when they see them, you should provide more evidence that your potentially interesting approach will provide solid results and verifiable progress.

I've just begun to consider whether a similar method might illuminate problems like self-enhancement, utility function discovery, and utility function renormalization (for concreteness, I plan to work with decision field theory).

Just begun to consider? This doesn't inspire confidence.

I have ideas about how CEV should work, and about what the true ontology is, and about the adjustments that the latter might require. These ideas are tentative, and open to correction, and the objective is to find out the facts, not just to insist on an opinion.

Philosophical intuition is a really bad determiner for research funds allocation. I have some ideas about ontology too, and I think they're very clever. They are related to the ideas of Paul Almond who is a very creative thinker and has a great intuition for Occamian reasoning. My metaphysical intuition finds it rather unlikely that string theory would be important for volition extrapolation: in an ensemble universe of hierarchal ontology, computation-specific physical laws are not as important as resolving confusion about things like self-representation and determining languages/prefixes for UTMs: problems that I and more importantly folk at the Singularity Institute are working on (and problems that people from the decision theory workshop sorta flirt with now and then).

I have ideas about how CEV should work, and about what the true ontology is, and about the adjustments that the latter might require.

Have you read and understood Tegmark's papers about the MUH? Have you read and understood Paul Almond? I'm skeptical that anyone could have ideas about what the true ontology is that bear on CEV. CEV is an impossible problem; impossible in the Yudkowskyan sense of the word, but still impossible. I've had many ideas about how to attack it. They don't work. It's hard. It's so hard it's impossible. When someone says that they have ideas about how CEV should work, I think, 'this person just doesn't understand how impossible CEV is'. Do you have evidence that my judgment is wrong enough that others should fund you?

I can't help but think that focusing on string theory is going down a wrong path, and I'm much more tempted to think that you haven't found the most important domains for research. Which isn't really fair, 'cuz it's damn near impossible to figure it out yourself, and SIAI folk aren't exactly open about AGI-related research, but there you go. Do you trust that your metaphysical intuition is better than everyone else's? Do you really think we should trust that? Unless you can make exceptionally strong arguments for doing so, asking for funding is premature.

I do wish more people were working more directly on Friendliness, especially as Eliezer is writing a book right now. But I don't think anyone can. I'm not sure Eliezer or Marcello can, either, because it's an impossible problem. But with very, very few exceptions, I don't think anyone else is anywhere close.

Added: When I get replies like the one I made above, it makes me really depressed, sometimes for days. Even if I think they're off-base and ill-founded, it feels like someone's personally attacking me for no good reason. I'm not really sure how to soften the blow... but I thought that such a comment needed to be made. I'm sorry.

Added again: Instead of just being sorry I decided to try to be a little more productive. Hopefully my new post will be at least a little helpful.

Comment author: NihilCredo 24 September 2010 03:45:59PM *  3 points [-]

My metaphysical intuition finds it rather unlikely that string theory would be important for volition extrapolation: in an ensemble universe of hierarchal ontology, computation-specific physical laws are not as important as resolving confusion about things like self-representation and determining languages/prefixes for UTMs: problems that I and more importantly folk at the Singularity Institute are working on (and problems that people from the decision theory workshop sorta flirt with now and then).

Flesch-Kincaid grade 37. Congratulations, I don't think many people regularly pull that off without deliberate intent.

Comment author: wedrifid 24 September 2010 06:12:21PM -1 points [-]

Wow. Most people can't pull that off even when trying!