Seems to me (but I am not an expert) that most of the brain functionality is not like this. Especially the parts related to thinking. Thinking is usually slow and needs to be learned -- which is the exact opposite of how the massively parallelized parts work.

I'm not an expert either, but from what I've read on the subject, most of the neocortex does work like this. The architecture used in the visual cortex is largely the same as that used in the rest of the cortex, with some minor variations. This is suggested by the fact that people who lose an area of their neocortex are often able to recover, with another area filling in for it. I'm on a phone, so I can't go into as much detail as I'd like, but I recommend investigating the work of Mountcastle, and more recently Markram.

Edit: On Intelligence by Jeff Hawkins explains this principle in more depth, it's an interesting read.

I would honestly be very surprised if you actually kept track of every visual detail verbally.

Sorry, I didn't really explain this very well - I'm not tracking them verbally. In fact, most of the time I don't tend to represent my thoughts verbally either (something else that seems to surprise people when we discuss how we think), they exist in the same state that my spacial representations of things do - at least until I need them in a verbal form (such as when I'm trying to decide on the wording of something I'm going to say or write). It's hard for me to describe the state my thoughts take to someone else, as there's no analogue outside of my own head. When I said:

I can remember details of appearance, but they take the form of 'has ~5cm, dark straight hair', rather than a picture of how said hair might look.

What I meant is that there are some sort of symbols in my mind that represent these concepts being activated, but in a more abstract way than by using the words that describe them, or by picturing the images that make me think of them. It feels a little like there's a more abstract layer that sits on top of my visual and verbal systems, and this is where I do my thinking and imagining. If I need to, I can 'bring my thought down' to these parts of my brain (like when I'm deciding what to write, or how to draw something), but it's not the default case.

It seems to me like your visual cortex is processing the "picture" but for some reason you aren't experiencing it directly...

I guess this could be a possibility, but I do experience the "picture" to a degree if I actually make an effort to visualise it, like when I'm trying to draw a scene from my imagination. It's not really anything like what I see when I'm actually looking at something, but I assume that's the case for everyone. For example, if I try to do a puzzle that involves picturing something from a different angle, I'm able to, but it takes a conscious effort. The process I use for this doesn't seem to get involved when I'm imagining a scene from a book (unless I'm doing something like trying to picture a scene from someone else's point of view, at which point I have to stop reading briefly while I build up a picture).

I also struggle to create mental imagery. In this scenario, I do imagine a scene, but it's not generally composed of imagery. I'm aware that this probably won't make sense to someone who has a primarily visual imagination, but when I'm imagining a scene like this, it's more like a network of facts in my head - I piece together the underlying concepts behind what's being described, rather than a visual representation of them.

For example, if I'm imagining a room full of people, I'll have a mental model of everyone's positions in the room, which I'll then update if the story mentions that someone is stood at the left of the room and I'm imagining them at the right. However, I don't have a picture of the room in my head while I'm doing this, there's no image of where the people are stood - it's just something I 'know'.

If I'm given a description of a person's appearance, I incorporate facts about their appearance in my mental model of them, but I still don't form a mental image representing them. If I was asked to draw someone that was described to me, I could attempt it (despite my poor drawing skills), but I would be converting my mental model into a picture at this stage - I wouldn't be drawing from a picture in my head. When I try to recall a fictional character, my mental model of them is overwhelmingly based on things like personality traits, and my perception of how their mind might work. I can remember details of appearance, but they take the form of 'has ~5cm, dark straight hair', rather than a picture of how said hair might look.