Roughly, the importance is that there's only two kinds of truly catastrophic mistakes that an AI could make, mistakes which manage to wipe out to whole planet in one shot and errors in modifying its own code. Everything else can be recovered from.
It could build and deploy an unfriendly AI completely different from itself.
I could come up with an argument for this falling into either category.
I'm claiming that the concept of self-modification is useless since it's a special case of engineering. We have to get engineering right, and if we do that, we'll get self-modification right. I'm struggling to interpret your statement so it bears on my claim. Perhaps you agree with me? Perhaps you're ignoring my claim? You don't seem to be arguing against it.
The scenario I proposed (creating a new UFAI from scratch) doesn't fit well into the second category (self-modification) because I didn't say the original AI goes away. After the misbegotten creation of the UFAI, you have two, the original failed FAI and the new UFAI.
Actually, the second category (bad self-modification) seems to fit well into the first category (destroying the planet in one go), so these two categories don't support the idea that self-modification is a useful concept.
Okay, I think I see what you mean about engineering and self-modification, but I don't think its particularly important, it appears you're thinking in terms of two concepts:
Self-modification: Anything the AI does to itself, for a fairly strict definition of 'itself', as in, the same physical object' or something like that.
Engineering: Building any kind of machine.
However, I think that when most FAI researchers talk about 'self-modification' they mean something broader than your definition, which would include building another AI of roughly equal or greater...
I don't know if this is a little too afar field for even a Discussion post, but people seemed to enjoy my previous articles (Girl Scouts financial filings, video game console insurance, philosophy of identity/abortion, & prediction market fees), so...
I recently wrote up an idea that has been bouncing around my head ever since I watched Death Note years ago - can we quantify Light Yagami's mistakes? Which mistake was the greatest? How could one do better? We can shed some light on the matter by examining DN with... basic information theory.
Presented for LessWrong's consideration: Death Note & Anonymity.