nyan_sandwich comments on Tiling Agents for Self-Modifying AI (OPFAI #2) - Less Wrong

55 Post author: Eliezer_Yudkowsky 06 June 2013 08:24PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (260)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 06 June 2013 08:08:29PM 2 points [-]

Based on Eliezer's recent comments, my impression is that Eliezer is not making such a case, and is rather making a case for the paper being of sociological/motivational value.

No, that's not what I've been saying at all.

I'm sorry if this seems rude in some sense, but I need to inquire after your domain knowledge at this point. What is your level of mathematical literacy and do you have any previous acquaintance with AI problems? It may be that, if we're to proceed on this disagreement, MIRI should try to get an eminent authority in the field to briefly confirm basic, widespread, and correct ideas about the relevance of doing math to AI, rather than us trying to convince you of that via object-level arguments that might not be making any sense to you.

By 'the relevance of math to AI' I don't mean mathematical logic, I mean the relevance of trying to reduce an intuitive concept to a crisp form. In this case, like it says in the paper and like it says in the LW post, FOL is being used not because it's an appropriate representational fit to the environment... though as I write this, I realize that may sound like random jargon on your end... but because FOL has a lot of standard machinery for self-reflection of which we could then take advantage, like the notion of Godel numbering or ZF proving that every model entails every tautology... which probably doesn't mean anything to you either. But then I'm not sure how to proceed; if something can't be settled by object-level arguments then we probably have to find an authority trusted by you, who knows about the (straightforward, common) idea of 'crispness is relevant to AI' and can quickly skim the paper and confirm 'this work crispifies something about self-modification that wasn't as crisp before' and testify that to you. This sounds like a fair bit of work, but I expect we'll be trying to get some large names to skim the paper anyway, albeit possibly not the Early Draft for that.

Comment author: [deleted] 06 June 2013 08:15:27PM 2 points [-]

I need to inquire after your domain knowledge at this point. What is your level of mathematical literacy and do you have any previous acquaintance with AI problems?

Quick Googling suggest someone named "Jonah Sinick" is a mathematician in number theory. It appears to be the same person.

Comment author: Eliezer_Yudkowsky 06 June 2013 08:19:28PM 1 point [-]

I really wish Jonah had mentioned that some number of comments ago, there's a lot of arguments I don't even try to use unless I know I'm talking to a mathematical literati.

Comment author: jklsemicolon 06 June 2013 08:45:22PM *  1 point [-]

It's mentioned explicitly at the beginning of his post Mathematicians and the Prevention of Recessions, strongly implied in The Paucity of Elites Online, and the website listed under his username and karma score is http://www.mathisbeauty.org.