(We should be done with the mathy posts, I think, at least for now. But forgive me if, ironically, I end up resorting to Rationality Quotes for a day or two. I'm currently at the AGI-08 conference, which, as of the first session, is not nearly so bad as I feared.)
Suppose I tell you: "It's the strangest thing: The lamps in this hotel have triangular lightbulbs."
You may or may not have visualized it—if you haven't done it yet, do so now—what, in your mind's eye, does a "triangular lightbulb" look like?
In your mind's eye, did the glass have sharp edges, or smooth?
When the phrase "triangular lightbulb" first crossed my mind—no, the hotel doesn't have them—then as best as my introspection could determine, I first saw a pyramidal lightbulb with sharp edges, then (almost immediately) the edges were smoothed, and then my mind generated a loop of flourescent bulb in the shape of a smooth triangle as an alternative.
As far as I can tell, no deliberative/verbal thoughts were involved—just wordless reflex flinch away from the imaginary mental vision of sharp glass, which design problem was solved before I could even think in words.
Believe it or not, for some decades, there was a serious debate about whether people really had mental images in their mind—an actual picture of a chair somewhere—or if people just naively thought they had mental images (having been misled by "introspection", a very bad forbidden activity), while actually just having a little "chair" label, like a LISP token, active in their brain.
I am trying hard not to say anything like "How spectacularly silly," because there is always the hindsight effect to consider, but: how spectacularly silly.
This academic paradigm, I think, was mostly a deranged legacy of behaviorism, which denied the existence of thoughts in humans, and sought to explain all human phenomena as "reflex", including speech. Behaviorism probably deserves its own post at some point, as it was a perversion of rationalism; but this is not that post.
"You call it 'silly'," you inquire, "but how do you know that your brain represents visual images? Is it merely that you can close your eyes and see them?"
This question used to be harder to answer, back in the day of the controversy. If you wanted to prove the existence of mental imagery "scientifically", rather than just by introspection, you had to infer the existence of mental imagery from experiments like, e.g.: Show subjects two objects and ask them if one can be rotated into correspondence with the other. The response time is linearly proportional to the angle of rotation required. This is easy to explain if you are actually visualizing the image and continuously rotating it at a constant speed, but hard to explain if you are just checking propositional features of the image.
Today we can actually neuroimage the little pictures in the visual cortex. So, yes, your brain really does represent a detailed image of what it sees or imagines. See Stephen Kosslyn's Image and Brain: The Resolution of the Imagery Debate.
Part of the reason people get in trouble with words, is that they do not realize how much complexity lurks behind words.
Can you visualize a "green dog"? Can you visualize a "cheese apple"?
"Apple" isn't just a sequence of two syllables or five letters. That's a shadow. That's the tip of the tiger's tail.
Words, or rather the concepts behind them, are paintbrushes—you can use them to draw images in your own mind. Literally draw, if you employ concepts to make a picture in your visual cortex. And by the use of shared labels, you can reach into someone else's mind, and grasp their paintbrushes to draw pictures in their minds—sketch a little green dog in their visual cortex.
But don't think that, because you send syllables through the air, or letters through the Internet, it is the syllables or the letters that draw pictures in the visual cortex. That takes some complex instructions that wouldn't fit in the sequence of letters. "Apple" is 5 bytes, and drawing a picture of an apple from scratch would take more data than that.
"Apple" is merely the tag attached to the true and wordless apple concept, which can paint a picture in your visual cortex, or collide with "cheese", or recognize an apple when you see one, or taste its archetype in apple pie, maybe even send out the motor behavior for eating an apple...
And it's not as simple as just calling up a picture from memory. Or how would you be able to visualize combinations like a "triangular lightbulb"—imposing triangleness on lightbulbs, keeping the essence of both, even if you've never seen such a thing in your life?
Don't make the mistake the behaviorists made. There's far more to speech than sound in air. The labels are just pointers—"look in memory area 1387540". Sooner or later, when you're handed a pointer, it comes time to dereference it, and actually look in memory area 1387540.
What does a word point to?
This is probably one the most thought-provoking comments section i've read on this site so far.
I never really thought how different people's visualizing (or lack thereof) could be. Specifically, I never thought some people couldn't visualize at all. I always kind of assumed that people visualized fairly similarly to me. Looking back, this was a naive and selfish view, but still, so much difference...
For example, I saw the man walking on the left side of the street. I was standing in the middle of the sidewalk, at roughly my real-life height. the man was shorter than me (im 190cm, this is usually a safe bet). The man was elderly and heavily hunched, using a cane as he walked away from me (he was moving slowly). he was wearing a faded brown suit and hat you might have seen a senior wear in the 60's or 70's (funny considering im 22, and was nowhere near alive in that time-frame. I blame movies for this classic small-town scene). The sidewalk had small trees with short metal fence-cages around them (whatever those things are called) by the road spaced evenly every couple buildings apart. there were cars parked (no details on the cars other then that they were all sedan-sized). The street was deserted other than the man. it was approximately 5-6pm in summer by the sunlight and warmth (i don't know how temperature sneaked in there). the man turns left into a drugstore. he did not open the door, it swung open for him by itself somehow (though there was a faint metallic bell-sound when the door opened, as you often hear when walking into a small store). I didn't see him go in. the scene ended before he passed out of vision behind the door (it opened on the near side from him).
I have no idea how someone could possible imagine a scene like this without actually knowing the mans orientation, or personal position. I literally cannot conceive this scene without being a part of it, and having that level of detail. maybe this is why I enjoy novels and gaming so much. Specifically fantasy novels and games. I feel like I'm literally a part of another world and living the adventure.
Granted, thinking further on my "vision" some very interesting things come to mind. The man walks slowly and in real life his walking should have taken 30-40 seconds. However in my scene it happened in a split second. I have no idea how my brain managed to compress time. and it doesn't feel like the scene jumped. it felt normal. Also there was no one else on the street that i visualized, but i somehow knew it was a semi-busy small town street, and there were some people walking around, even though i could not see them.
Its actually mildly unnerving how i can visualize the scene with no other people there, and yet be absolutely certain that the scene i visualized had many other people too. And even more unnerving how I managed to watch 30-40 secs of footage in normal speed and have it happen in 1-2 secs. Anyone know how these things happen in the brain? I'd love to know more about these phenomena.