I'm afraid I'm a bit slow on the uptake here. Why does this require solipsism?

Well, there's exactly one being in existence that you know for sure is conscious and experiences qualia: yourself. You suspect that other beings (such as myself) are conscious as well, based on available evidence, though you can't be sure. This, by itself, is not a problem. What evidence could you use, though ? Here are some options.

You could say, "I think other humans are conscious because they have the same kind of brains that I do", but then you'd have to exclude other potentially conscious beings, such as aliens, uploaded humans, etc., and I'm not sure if you want to go that route (let me know if you do). In addition, it's still possible that any given human is not a human at all, but one of those perfect emulator-androids, so this doesn't buy you much.

You could put the human under a brain scanner, and demonstrate that his brain states are similar to your own brain states, which you have identified as contributing to consciousness. If you could do that, though, then you would've reduced consciousness down to physical brain states, and the problem would be solved, and we wouldn't be having this conversation (though you'd still have a problem with aliens and uploaded humans and such).

You could also observe the human's behavior, and say, "this person behaves exactly as though he was conscious, therefore I'm going to assume that he is, until proven otherwise". However, since you postulate the existence of androids/zombies/etc. that emulate consciousness perfectly without experiencing, you can't rely on behavior, either.

Basically, try as I might, I can't think of any piece of evidence that would let you distinguish between a being -- other than yourself -- who is consciousness and experiences qualia, and a being who pretends to be conscious with perfect fidelity, but does not in fact experience qualia. I don't think that such evidence could even exist, given the existence of perfect zombies (since they would be imperfect if such evidence existed). Thus, you are forced to conclude that the only being who is conscious is yourself, which is a kind of solipsism (though not the classic, existential kind).

Similarly, I sometimes just get a "feel" for what someone's emotional state is without really being able to pinpoint why I get that impression. But as long as we're talking about a generalized sense of "behavior" that includes cues that go unnoticed by the conscious mind, then sure!

It seems like we agree on this point, then -- yey ! Of course, I would go one step further, and argue that there's nothing special about our subconscious mind. We know how some parts of it work, we have mapped them down to physical areas of the brain, and our maps are getting better every day.

I think you're pointing out that it'd be very odd to have a quale-free but otherwise perfect simulation of a human mind. I agree, that would be odd.

I don't just think it would be odd, I think it would be logically inconsistent, as long as you're willing to assume that people other than yourself are, in fact, conscious. If you're not willing to assume that, then you arrive at a kind of solipsism, which has its own problems.

But what's even more odd is that even though we can be extremely confident that there's some mechanism that goes from firing neurons to qualia, we have no clue what it could be.

Right, which is why I reject the existence of qualia as an independent entity altogether. As per your magnetism analogy:

"Surely, objects can't act at a distance without a medium, so there must be some kind of stuff going on between the magnets to pull them toward one another." Sure, that's close enough, but if you focus on building more and more powerful microscopes to try to find that medium, you'll be SOL.

Right, and the problem here is not that your microscopes aren't powerful enough, but that your very idea of a magnetic attraction medium is flawed. In reality, there are (probably) no such things as "magnets" at all; there are just collections of waveforms of various kinds (again, probably). You choose to call some of them "magnets" and some others "apples", but those words are just grossly simplified abstractions that you have created in order to talk about the world -- because if you had to describe every single quark of it, you'd never get anywhere.

Similarly, "qualia" and "consciousness" are just abstractions that you'd created in order to talk about human brains -- including your own brain. I understand that you can observe your own consciousness "from the inside", which is not true of magnets, but I don't see this as an especially interesting fact. After all, you can observe gravity "from the inside", as well (your body is heavy, and tends to fall down a lot), but that doesn't mean that your own gravity is somehow different from my gravity, or a rock's gravity, because as far as gravity is concerned, you aren't special.

If this fuzzily stated guess is in the right ballpark, then the reason consciousness seems like such a hard problem is that we can't ever pin down a part of the brain that is the "seat of consciousness", nor can we ever say exactly when a signal from the optic nerve turns into vision.

I don't think that we need to necessarily pin down a single part of the brain that is the "seat of consciousness". We can't pin down a single part that constitutes the "seat of vision", either, but human vision is nonetheless fairly well understood by now. The signal from the optic nerve is just part of the larger mechanism which includes the retina, the optic nerve, the visual cortex, and ultimately a large portion of the brain. There's no point at which electrochemical signals turn into vision, because these signals are a part of vision. Similarly, there isn't a single "seat of blood flow" within the human body, but blood flow is likewise fairly well understood.

Similarly, we can't just "remove consciousness", although we can remove parts of it (e.g., cutting out the narrator or messing with the coordination, as in meditation or alcohol).

I'm not sure I follow your reasoning here. What do you mean by "removing consciousness" and "cutting out the narrator", and why is it important ? Drunk (or meditating) people are still conscious, after a fashion.