Or is it more IQ being necessary for efficient self-modification?
Sounds like false dilemma if IQ is one of those things that can be modified. :D
To unpack the word, IQ approximately means how complex concepts can you "juggle" in your head. Without enough IQ, even if you had an easy computer interface to modify your own brain, you wouldn't understand what exactly you should do to achieve your goals (because you wouldn't sufficiently understand the concepts and the possible consequences of the changes). That means, you would be making those changes blindly... and you could get lucky and hit the path where your IQ increases so then you can start making reliably the right steps, or you could set yourself on a way towards some self-destructive attractor.
As a simple example, a stupid person could choose to press a button that activates their reward center... and would keep doing that until they die from starvation. Another example would be self-modification where you lose your original goals, or lose the will to further self-improve, etc. This does not have to happen in one obvious step, but could be a subtle cumulative consequence of many seemingly innocent steps. For example, a person could decide that being fit is instrumentally useful for their goals, so they would self-modify to enjoy exercise, but would make a mistake of modifying themselves too much, so now they only want to exercise all day long, and no longer care about their original goals. Then they would either stop self-modifying, or self-modify merely to exercise better.
It also depends on how complex would be the "user interface to modify your own brain". Maybe IQ 120 would not be enough to understand it. Maybe even IQ 200 wouldn't. You could just see a huge network of nodes, each connected to hundreds of other nodes, each one with insanely abstract description... and either give up, or start pushing random buttons and most likely hurt yourself.
So basicly the lowest necessary starting IQ is the IQ you need to self-modify to safely enough increase your IQ. This is a very simple model which assumes that if IQ N allows you to get to IQ N+1, then IQ N+1 probably allows you to get to IQ N+2. The way does not have to be this smooth; there may be a smooth increase up to some level, which then requires a radical change to overcome; or maybe the intelligence gains at each step will decrease.
Imagine trying to compete with Stalin for the job of being Lenin's successor. What traits you need for it?
Courage, social skills, ability to understand how politics really works. You should probably start in some position in army or secret service, some place where you can start building your own network of loyal people, without being noticed by Stalin. Or maybe you should start as a crime boss in hiding, I don't know.
To unpack the word, IQ approximately means how complex concepts can you "juggle" in your head
I think I agree with this, this is why I don't understand how can intelligence be defined as goal-achieving ability. When I am struggling with the hardest exercises on the Raven test, what I wish I had more is not some goal-achieving power but something far simpler, something akin to short term memory. So when I wish for more intelligene, I just wish for a bit more RAM in short-term memory, so more detailed, more granular ideas can be uploaded into tumble space. No idea why should it mean goal-achieving or optimizing ability. And for AI IQ sounds like entirely hardware...
Edit: Some people have misunderstood my intentions here. I do not in any way expect this to be the NEXT GREAT IDEA. I just couldn't see anything wrong with this, which almost certainly meant there were gaps in my knowledge. I thought the fastest way to see where I went wrong would be to post my idea here and see what people say. I apologise for any confusion I caused. I'll try to be more clear next time.
(I really can't think of any major problems in this, so I'd be very grateful if you guys could tell me what I've done wrong).
So, a while back I was listening to a discussion about the difficulty of making an FAI. One of the ways that was suggested to circumvent this was to go down the route of programming an AGI to solve FAI. Someone else pointed out the problems with this. Amongst other things one would have no idea what the AI will do in pursuit of its primary goal. Furthermore, it would already be a monumental task to program an AI whose primary goal is to solve the FAI problem; doing this is still easier than solving FAI, I should think.
So, I started to think about this for a little while, and I thought 'how could you make this safer?' Well, first of, you don't want an AI who completely outclasses humanity in terms of intellect. If things went Wrong, you'd have little chance of stopping it. So, you want to limit the AI's intellect to genius level, so if something did go Wrong, then the AI would not be unstoppable. It may do quite a bit of damage, but a large group of intelligent people with a lot of resources on their hands could stop it.
Therefore, what must be done is that the AI cannot modify parts of its source code. You must try and stop an intelligence explosion from taking off. So, limited access to its source code, and a limit on how much computing power it can have on hand. This is problematic though, because the AI would not be able to solve FAI very quickly. After all, we have a few genius level people trying to solve FAI, and they're struggling with it, so why should a genius level computer do any better. Well, an AI would have fewer biases, and could accumulate much more expertise relevant to the task at hand. It would be about as capable as solving FAI as the most capable human could possibly be; perhaps even more so. Essentially, you'd get someone like Turing, Von Neumann, Newton and others all rolled into one working on FAI.
But, there's still another problem. The AI, if left for 20 years working on FAI for 20 years let's say, would have accumulated enough skills that it would be able to cause major problems if something went wrong. Sure, it would be as intelligent as Newton, but it would be far more skilled. Humanity fighting against it would be like sending a young Miyamoto Musashi against his future self at his zenith i.e. completely one sided.
What must be done then, is the AI must have a time limit of a few years (or less) and after that time is past, it is put to sleep. We look at what it accomplished, see what worked and what didn't, and boot up a fresh version of the AI with any required modifications, and tell it what the old AI did. Repeat the process for a few years, and we should end up with FAI solved.
After that, we just make an FAI, and wake up the originals, since there's no point in killing them off at this point.
But there are still some problems. One, time. Why try this when we could solve FAI ourselves? Well, I would only try and implement something like this if it is clear that AGI will be solved before FAI is. A backup plan if you will. Second, what If FAI is just too much for people at our current level? Sure, we have guys who are one in ten thousand and better working on this, but what if we need someone who's one in a hundred billion? Someone who represents the peak of human ability? We shouldn't just wait around for them, since some idiot would probably just make an AGI thinking it would love us all anyway.
So, what do you guys think? As a plan, is this reasonable? Or have I just overlooked something completely obvious? I'm not saying that this would by easy in anyway, but it would be easier than solving FAI.