I'm currently testing a promising direction for a possible collection of units at CFAR. (For those who have attended some CFAR events or test sessions, this is a collection of refinements to the fudoshin/"panic" unit.) I've hit on what I think is a key puzzle whose answer might unlock a lot of the emerging art of rationality. I - and possibly most people here, eventually - would very much appreciate any insight you have to share.
The puzzle is how thought incubation works, ideally expressed in terms of neural systems or neuroanatomical structures. I'll first explain the phenomenon and then suggest the general reference class from which I'm hoping to get an answer.
The Phenomenon:
Mathematicians frequently report that often one of the most helpful things they can do to solve a problem they're stuck on is step away from it. Jacques Hadamard (1949) examined his own experiences and also talked to many of his colleagues to work out what the common structure of this experience was, and determined that there seems to be a fairly predictable sequence to it:
(1) Intensely focus on the problem, working through every permutation you can think of that's likely to produce an answer.
(2) Walk away from the problem and think about something else.
(3) The magic genie in your head might eventually, and often unexpectedly, yell a possible insight into your awareness.
For instance, Henri Poincaré reported struggling to work on Fuchsian functions over the course of several weeks and then being forced to walk away from the proof he had been stuck on due to a planned vacation. One day he was stepping onto a bus with his mind certainly not on mathematics, and suddenly the key insight he needed to finish the proof appeared in his mind. It was as though a part of his mind had been secretly working on the problem and then brought the finished product into his awareness. In this particular case it also came with a feeling of total confidence that verification would pan out (although Hadamard notes that the validation step after the insight is still essential because sometimes that feeling of total confidence is mistaken).
I definitely relate to this from when I was working on graduate mathematics. However, it also pattern-matches with other mental phenomena that are much more common. For instance, sometimes I think I know what a person's name is, but struggle as I might I can't quite remember it - and then a few minutes later after I've given up remembering the name the answer loudly announces itself, often quite out-of-context. Or when I'm trying to figure out a way of improving a throw in martial arts and then find the answer suddenly dawning on me at a random time.
I'm under the impression that this is a fairly universal kind of experience. I suspect you can think of examples in your own life where this has happened. ("Oh, now I remember where I put those keys!")
Reference Class for an Explanation:
I'm going to offer some overly simplistic examples of the kind of explanation I'm looking for. In this case, I think overly simplistic might be okay since I'm just trying to get a reasonable handle on how to munchkin the interaction between a few different neural systems. If it turns out that a more detailed and technically correct version is important, I'll probably dig into it (pending the VOI versus cost-of-information comparison).
There seems to be some evidence that one of the reasons children are as impulsive as they are is that they haven't yet developed their prefrontal cortices (PFCs) to the degree adults have. The prefrontal cortex seems to do at least two things: (1) hold long-term goals in mind and (2) engage executive function (i.e., halt orders on impulses, typically ones that don't match up with the long-term goals). This neuroanatomical structure seems to continue growing until sometime in one's early 20s - which might be why we also find that teenagers typically have less impulse control than twentysomethings but more than middle-schoolers, whereas we don't find such a clear distinction between twentysomethings and thirtysomethings. (Yes, this could also or even instead be cultural. I know it's complicated.) Incidentally, I understand that the PFC is also one of the neural structures most deactivated by alcohol - although my impression is that it shuts down the long-term goals thing and not the executive function. (This is based on my and others' experience that precommitment works perfectly well. It seems to me that saying things like "I couldn't help myself because I was drunk!" is more a social excuse than an actual explanation. But I'm only around 65% confident of this as a general claim.)
On a related note, it would seem that there's something in the same rough space as theory of mind that goes beyond the ability to pass the false-belief test. According to Rebecca Saxe, the capacity for empathy seems to come from a particular bit of the brain that doesn't finish growing until the mid-20s. Saxe also provides some evidence that a sufficiently strong and precisely directed magnet can basically deactivate that part of one's theory of mind temporarily. It seems quite plausible to me (though I really don't know) that activation of the sympathetic nervous system (SNS), such as in fight-or-flight reactions, decreases activation of this empathy part of the brain. This might be why, in a perceived crisis, some people switch to an almost tool-like view of others (e.g., knowing that overcoming the bystander effect requires pointing at a specific person and saying "You! Call 911!" but not really getting a sense in that moment of what that person's experience is like to be so singled out).
I'm quite aware that much of the above is speculation. I think speculation is fine, but having it grounded in some actual known neuroscience is ideal. That would give me something to dig into. E.g., if there's some reason to believe that this phenomenon is related to the enteric nervous system, I can start digging into the literature on that system to better understand how to munchkin its interactions with the (rest of the) autonomic nervous system.
An example of something outside the reference class I'm looking for is a "little man in the subconscious" explanation. I first read about this about twenty years ago as a model for how mental incubation works: you concentrate on a problem in order to communicate to a little man in your subconscious what you want to have done, and then you stop talking to him so he can go do what you just told him to do. Then he comes back with an answer once he's done, without regard to what you're doing when he's done. I agree that this seems to be a reasonable metaphor for what's going on, but it doesn't tell me for instance why the "little man" seems to respond so much more to SNS activity than parasympathetic activity, or why he can't go do his job once he has the instructions even if we continue to think about the problem.
More generally, psychodynamic "explanations" are unlikely to be helpful here. Talking about this as the "domain of the iNtuiting function" in reference to Jungian psychodynamic theory or Myers-Briggs won't tell me hardly anything about how this relates to stress oscillation.
So... Any suggestions about what this mysterious "little man" might actually be made of?
I think the answer is hiding in this bit: "In this particular case it also came with a feeling of total confidence that verification would pan out (although Hadamard notes that the validation step after the insight is still essential because sometimes that feeling of total confidence is mistaken)."
I'm going to guess that the class of problems this work on is where the person knows somehow that there IS an answer. Almost as if they see the pattern from the outside yet haven't just worked out the details. This would explain why when they get the answer they 'know' it's correct. It would also explain the hard work AND the frustration of not getting it. After all, if you didn't intuit that there was an answer, if on some level you didn't see it, well then you wouldn't feel frustrated at not getting it!
And so the question then is, if you know the answer, if you can see it, well then why the heck can't you know it?!
For that I think that sometime it takes some time to fully experience what you see and know. For it to open up completely from implicit body knowledge to explicit cognition.
Now if you don't think about it at all, when then it's dormant. It's not on the table. At the same time, if you work on it too hard you very very often block yourself by 'going too fast' with your attempts to 'get it out there already'.
When you work on it, then stop, you have put it on the table and at the same time you aren't getting yourself stuck in a rut by trying too hard.
As for what is actually happening when you aren't thinking about it, well at all times you're implicitly holding in mind many many things. Most are invisible to you (the fact that you live in country X and have family Y and have profession Z). You act on those implicit contexts, you must be 'making contact' with them, yet they are invisible (do the fish know of the water they swim in).
Added to that are the implicit stuck things in your life. Like knowing that the whole thing with the landlord is not resolved... Even when you aren't thinking about it, you feel it. You sense it. Walking around during the day with that issue 'on the table' compared to walking around without it is a completely different experience.
A good way to experience this experientially in less then 5 minutes is to ask yourself "What is not perfect in my life?" When you get the answer, write it down and then say "And X isn't perfect, and without X, what's not perfect in my life?" Keep going till it stops. You'll know when to end... :)
When you do that exercise you FEEL the body sense of layers on implicit stuff that you're holding come off, if even for a moments.
When you have a difficult problem that you KNOW is solvable and you can't find it, well you feel that. All day. And that's the implicit processing going on.
(Let us know how the exercise went).