= 27d567bc2d48d9f40e9ccc20ea370b15)
whpearson comments on Open Thread: February 2010, part 2 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (857)
I've been wondering what the existance of Gene Networks tells us about recursively self improving systems. Edit: Not that self-modifying gene networks are RSIS, but the question is "Why aren't they?" In the same way that failed attempts at flying machines tell us something, but not much, about what flying machines are not. End Edit
They are the equivalent of logic gates and have the potential for self-modification and reflection, what with DNAs ability to make enzymes that chop itself up and do so selectively.
So you can possibly use them as evidence that low-complexity, low-memory systems are unlikely to RSI. How complex they get and how much memory they have, I am not sure.
I wonder if the distinction between self-modification and recursive self-improvement is one of those things that requires a magic gear to get, and otherwise can't be explained by any amount of effort.
I understand there is a distinction. Would you agree that RSI systems are conceptually a subset of self-modifying (SM) systems? One that we don't understand what exact properties make a SM system one that will RSI. Could you theoretically say why EURISKO didn't RSI?
I was interested in how big a subset. The bigger it is the more dangerous, the more easily we will find it.
Sure. In fact, some of the Lenat quotes on LW even tell you why.
As a hack to defeat 'parasitic' heuristics, Lent (& co.?) put into Eurisko a 'protected kernel' which couldn't be modified. This core was not good enough to get everything going, dooming Eurisko from a seed AI perspective, and the heuristics never got anywhere near the point they could bypass the kernel. Eurisko was inherently self-limited.
It seems to me that for SM to become RSI, the SM has to able to improve all the parts of the system that are used for SM, without leaving any "weak links" to slow things down. Then the question is (slightly) narrowed to what exactly is required to have SM that can improve all the needed parts.