So here's your homework problem: What kind of cognitive algorithm, as felt from the inside, would generate the observed debate about "free will"? - Dissolving the Question
Edit: In the linked post by Eliezer, he shifts the debate around free will and determinism. Instead of asking which is true, or trying to resolve or dissolve the philosophical question, he asks a different but related question about human psychology. Why does it feel like it makes sense to have this philosophical debate in the first place? What is it about our psychology that makes positions vis-a-vis free will and determinism seem so compelling? It is this question that I have attempted to answer here.
Put a glass of water in front of yourself. Now, make a decision: take a drink, or do not take a drink. Or at least imagine doing so.
When you've made your decision and acted on it, get very clear on which option you chose. Perhaps you should write it down, or at least make a mental note. Do not proceed until you've come to this point.
Now, you have a memory, one in which you seem to have made a decision of your own free will. Why does it feel that way? And why might somebody else argue that, no, in fact, you were fated to make that decision?
Let's consider the one or two main choices you made for this exercise. The first choice was whether or not to comply with the instructions. If you did, the second choice was whether or not to take a drink. Of course, there were other choices: how exactly to carry this out (in your imagination or in real life? with water, or coffee?). But we'll ignore them for now.
The process of making these major decisions took place slowly enough that you have some felt memory of the psychological operations you experienced in the course of making them. You read the instructions. Then, you may have split the future into two imagined alternatives. Some mysterious further operations that you'd have a very hard time putting into words took place. Your subconscious mind was playing a role too, but that's beyond your comprehension. But you clearly remember taking the physical actions to bring about one world state or the other.
Some aspects of this process of thought and action took place slowly and consciously enough to leave an accessible memory. Others were too fast, or two unconscious, to leave a trace.
The words of the instructions, the sense of splitting the world into two options (or not doing so), and the physical action (or lack thereof) all happened slowly enough to create a memory.
When you access that memory and reconstruct for yourself your past experience, you imagine yourself conceiving of two options - to comply with or to bypass the instructions, to drink or not to drink - and a sensation of selecting, then acting upon one of the options.
What you do not remember, because it happened too quickly or unconsciously, was the process by which that selection occurred. Or the operations underneath that. There is a physical sensation of volition that some part of your brain produces, which is perhaps a purely somatosensory feeling. Moving your arm (or not) when no external force is yanking or constraining your body physically feels distinct from being acted upon by an external force. That somatosensory sensation is the feeling of volition that you find associated with this decision in your memory.
The reason why it feels that you have made this decision of your own "free will" is that you cannot detect the overwhelming evidence against it. The mysterious, rapid, unconscious core brain activity that structures your conscious qualitative experience is unavailable to you. You have learned by imitation to describe certain somatosensory sensations in terms like will and volition. The way the instructions were written demands that you describe this reconstruction of memory in terms of "choice."
The counterargument of determinism arises because we can observe ourselves sometimes experiencing a lack of the familiar association between the choice-making memory and the somatosensation of volition. At times, you remember making a conscious choice, but did not physically feel like you made a decision. At other times, you feel a sensation like volition, but do not remember making a conscious choice. These disturbing disconnects provide some of the missing evidence that an unconscious process is driving your psychological operations, your motor impulses, your somatosensations, and the formation of memory.
Furthermore, you realize that your ability to give an account of precisely which psychological operations led to a decision rapidly bottom out. For any given decision, after saying, "and why did you think that?" a finite (and probably small) number of times, you lose the ability to answer. Yet you infer that there must be a reason why. And you realize that reason is unavailable to conscious inspection, yet must have originated in your brain.
To really solve this homework question, you'd probably also need to explain the emotional valence of the debate, why people speculate on their own psychology in general, and why it is part of our culture. But I think this is a decent account of the mental mechanisms that produce an intuitive notion of free will, and also some evidence for an argument against it.
I do like your framing of "sophisticated naive introspection," and I think it makes sense as an object of study. That seems like a nice generalization of the program of study that Eliezer is calling for in his post.
However, it seems to me that your distinction between an "abstract model of the world that our brains build" vs. an "accurate view," or the idea that there's a categorical difference between introspection and what we might call "actual psychology," seems wrong to me.
Everything we use to study the brain, including naive introspection, is an abstract model of the world. Psychology refines, and sometimes directly contradicts the findings of naive introspection; the same is true of the relationship between naive and scientific physics. However, in both cases, intuition and reality line up pretty nicely, at least for some subset of situations. For that limited set of use-cases, the difference between naive and scientific approaches to the discipline isn't worth worrying about in practical terms.
One of the unstated assumptions of my argument here is that the abstract psychological model afforded by naive introspection is, in this case, adequate to at least make some headway on this problem. It wouldn't be nearly adequate to resolve it - I haven't even really scratched the surface. But that wasn't my goal here. I just wanted to give the best quick account I could of why "free will" and "determinism" seem to make psychological sense, based on my own intuitions and thoughts and given a short investment of time.