I'll bet you £100 that whatever conclusions the SIAI eventually draws about self modification will apply just as well to all kinds, I really cannot see how a silly distinction like the one you are making would find its way into a mathematical proof.
We're certainly agreed on that. I'm willing to go further -- I believe any mathematical conclusions that apply to self-modification (your definition) will apply to all possible actions. I don't think your definition carves out a part of the world that has any usefully special properties.
Worse, [self-modification interpreted as requiring a modification to the entity taking action] is not even well-defined, if an AI copies itself onto another computer, and alters the copy, is that self-modification? If it modifies itself, but keeps a copy of its old code around, is that self-modification? Where do you draw the line between the two?
Agreed.
I don't think your definition is well-defined either. Where's the important line between self-modification and making a toaster?
We appear to have no useful definition for the word. Time to stop using it, IMO.
We're certainly agreed on that. I'm willing to go further -- I believe any mathematical conclusions that apply to self-modification (your definition) will apply to all possible actions. I don't think your definition carves out a part of the world that has any usefully special properties.
I disagree. "An ideal CDT agent that anticipates facing only action-determined problems will always choose not to self modify" is true "An ideal CDT agent that anticipates facing only action-determined problems will always choose not to do anything" i...
I don't know if this is a little too afar field for even a Discussion post, but people seemed to enjoy my previous articles (Girl Scouts financial filings, video game console insurance, philosophy of identity/abortion, & prediction market fees), so...
I recently wrote up an idea that has been bouncing around my head ever since I watched Death Note years ago - can we quantify Light Yagami's mistakes? Which mistake was the greatest? How could one do better? We can shed some light on the matter by examining DN with... basic information theory.
Presented for LessWrong's consideration: Death Note & Anonymity.