derekz comments on Let's reimplement EURISKO! - Less Wrong

19 Post author: cousin_it 11 June 2009 04:28PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (151)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 11 June 2009 08:27:01PM 7 points [-]

Not exactly, Thom. Roughly, for FAI you need precise self-modification. For precise self-modification, you need a precise theory of the intelligence doing the self-modification. To get to FAI you have to walk the road that leads to precise theories of intelligence - something like our present-day probability theory and decision theory, but more powerful and general and addressing issues these present theories don't.

Eurisko is the road of self-modification done in an imprecise way, ad-hoc, throwing together whatever works until it gets smart enough to FOOM. This is a path that leads to shattered planets, if it were followed far enough. No, I'm not saying that Eurisko in particular is far enough, I'm saying that it's a first step along that path, not the FAI path.

Comment author: derekz 11 June 2009 09:52:41PM 4 points [-]

Perhaps a writeup of what you have discovered, or at least surmise, about walking that road would encourage bright young minds to work on those puzzles instead of reimplementing Eurisko.

It's not immediately clear that studying and playing with specific toy self-referential systems won't lead to ideas that might apply to precise members of that class.

Comment author: Eliezer_Yudkowsky 11 June 2009 11:01:56PM 7 points [-]

I've written up some of the concepts of precise self-modification, but need to collect the posts on a Wiki page on "lawfulness of intelligence" or something.

Comment author: Kevin 23 January 2010 09:01:55AM 4 points [-]

Any of these posts ever go up?

Comment author: Zack_M_Davis 23 January 2010 03:49:42PM 3 points [-]