(We should be done with the mathy posts, I think, at least for now. But forgive me if, ironically, I end up resorting to Rationality Quotes for a day or two. I'm currently at the AGI-08 conference, which, as of the first session, is not nearly so bad as I feared.)
Suppose I tell you: "It's the strangest thing: The lamps in this hotel have triangular lightbulbs."
You may or may not have visualized it—if you haven't done it yet, do so now—what, in your mind's eye, does a "triangular lightbulb" look like?
In your mind's eye, did the glass have sharp edges, or smooth?
When the phrase "triangular lightbulb" first crossed my mind—no, the hotel doesn't have them—then as best as my introspection could determine, I first saw a pyramidal lightbulb with sharp edges, then (almost immediately) the edges were smoothed, and then my mind generated a loop of flourescent bulb in the shape of a smooth triangle as an alternative.
As far as I can tell, no deliberative/verbal thoughts were involved—just wordless reflex flinch away from the imaginary mental vision of sharp glass, which design problem was solved before I could even think in words.
Believe it or not, for some decades, there was a serious debate about whether people really had mental images in their mind—an actual picture of a chair somewhere—or if people just naively thought they had mental images (having been misled by "introspection", a very bad forbidden activity), while actually just having a little "chair" label, like a LISP token, active in their brain.
I am trying hard not to say anything like "How spectacularly silly," because there is always the hindsight effect to consider, but: how spectacularly silly.
This academic paradigm, I think, was mostly a deranged legacy of behaviorism, which denied the existence of thoughts in humans, and sought to explain all human phenomena as "reflex", including speech. Behaviorism probably deserves its own post at some point, as it was a perversion of rationalism; but this is not that post.
"You call it 'silly'," you inquire, "but how do you know that your brain represents visual images? Is it merely that you can close your eyes and see them?"
This question used to be harder to answer, back in the day of the controversy. If you wanted to prove the existence of mental imagery "scientifically", rather than just by introspection, you had to infer the existence of mental imagery from experiments like, e.g.: Show subjects two objects and ask them if one can be rotated into correspondence with the other. The response time is linearly proportional to the angle of rotation required. This is easy to explain if you are actually visualizing the image and continuously rotating it at a constant speed, but hard to explain if you are just checking propositional features of the image.
Today we can actually neuroimage the little pictures in the visual cortex. So, yes, your brain really does represent a detailed image of what it sees or imagines. See Stephen Kosslyn's Image and Brain: The Resolution of the Imagery Debate.
Part of the reason people get in trouble with words, is that they do not realize how much complexity lurks behind words.
Can you visualize a "green dog"? Can you visualize a "cheese apple"?
"Apple" isn't just a sequence of two syllables or five letters. That's a shadow. That's the tip of the tiger's tail.
Words, or rather the concepts behind them, are paintbrushes—you can use them to draw images in your own mind. Literally draw, if you employ concepts to make a picture in your visual cortex. And by the use of shared labels, you can reach into someone else's mind, and grasp their paintbrushes to draw pictures in their minds—sketch a little green dog in their visual cortex.
But don't think that, because you send syllables through the air, or letters through the Internet, it is the syllables or the letters that draw pictures in the visual cortex. That takes some complex instructions that wouldn't fit in the sequence of letters. "Apple" is 5 bytes, and drawing a picture of an apple from scratch would take more data than that.
"Apple" is merely the tag attached to the true and wordless apple concept, which can paint a picture in your visual cortex, or collide with "cheese", or recognize an apple when you see one, or taste its archetype in apple pie, maybe even send out the motor behavior for eating an apple...
And it's not as simple as just calling up a picture from memory. Or how would you be able to visualize combinations like a "triangular lightbulb"—imposing triangleness on lightbulbs, keeping the essence of both, even if you've never seen such a thing in your life?
Don't make the mistake the behaviorists made. There's far more to speech than sound in air. The labels are just pointers—"look in memory area 1387540". Sooner or later, when you're handed a pointer, it comes time to dereference it, and actually look in memory area 1387540.
What does a word point to?
Much of this discussion seems to be people expressing differences in their own internal processes when it comes to visualization, reading, etc. This seems to me to be directly connected with a concept it took me many years to learn and which still feels unnatural to me: not all people think the same way. Even if one assumes that people all have identical brains at birth (which is not true, but useful for the sake of this argument) our brains start with a vast number of connections, and then as we age and gain experience those connections are pruned, so that we retain only the connections we use. The parts of our brain we use to process specific types of data would then be reinforced to process that data in the future, and our thinking would diverge.
This is a topic particularly prone to argument because whether or not other people think the same way we do is a difficult concept to think about, but the comments on this article are strong evidence that people at least perceive their mental processes differently. I cannot think of a good experiment to validate our different internal perceptions, but I think it would be overhasty to try to denounce other people's perceptions of how they process certain data such as visualizations.
I personally cannot picture my mother's face. I cannot picture my childhood bedroom. I can describe them in my head with words, but that is all. I can, however, rearrange the furniture in a room in my head and be sure that it will fit, or perceive the three dimensional shape described by a calculus problem. It feels to me like proprioception. I cannot say that with certainty, as I don't have a test for it, but the evidence of my own perceptions, and the evidence of, for example, the visualize-a-tiger experiment (in which some people had visualized it down to the stripes and others were taken aback by the question), both contribute to my belief that the parts of my brain I use to perceive certain data are not necessarily the same as yours, and that different mental abilities, such as visualization, are found at different strengths in different people.
Yep - a.k.a. the typical mind fallacy.