I'm having a really hard time modeling your thought process. Like, I don't know what is generating the things that you are saying; I am confused.
I'm not sure what you mean by inner vs. outer view.
Well, IQ tests test lots of things.
Is IQ the size of the door on the shop that determines how big a machine can be brought in for breaking down?
This seems like a good metaphor for working memory, and even though WM correlates with IQ, it's also just one component.
I don't really get what you mean when you say that it's important how we visualize it.
Well, if you take, say, AIXI, which sounds like this sort of hypothesis-testing-optimizer-type AI that you're talking about, AIXI takes an action at every timestep, so if you consider a hypothetical where AIXI can exist and still be unbounded, or maybe a computable approximation in which it has a whole hell of a lot of resources and a realistic world model, one of those actions could be human natural language if it happened to be the action that maximized expected reward. So I'd say that you're anthropomorphizing a bit too much. But AIXI is just the provably-best jack-of-all-trades; from what I understand there could be algorithms that are worse than AIXI in other domains but better in particular domains.
I think the keyword to my thought process is anthropomorphizing. The intuitive approach to intelligence is that it is a a human characteristic, almost like handsomeness or richness. Hence the pop culture AI is always an anthropomorphic conversation machine from Space Odyssey to Matrix 3 to Knight Rider. For example, it should probably have a sense of humor.
The approach EY/MIRI seems to take is to de-anthropomorphize even human intelligence as an optimization engine. A pretty machine-like thing. This is what I mentioned that I am not sure I can buy it, but...
If it's worth saying, but not worth its own post (even in Discussion), then it goes here.
Notes for future OT posters:
1. Please add the 'open_thread' tag.
2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)
3. Open Threads should be posted in Discussion, and not Main.
4. Open Threads should start on Monday, and end on Sunday.