We refer to things with signs. The signs don't have to be visual representations. We can think about things by employing the signs which refer to them. What makes the sign for (say) countable infinity refer to it is the way that the sign is used in a mathematical theory (countable infinity being a mathematical concept). Learn the math, and you will learn the concept.
Compare to this: you probably cannot visualize the number 845,264,087,843,113. You can of course visualize the sign I just now wrote for it, but you cannot visualize the number itself (by, for example, visualizing a large bowl with exactly that number of pebbles in it). What you can do is visualize a bowl with a vast number of pebbles in it, while thinking the thought, "this imagined bowl has precisely 845,264,087,843,113 pebbles in it." Here you would be relying entirely on the sign to make your mental picture into a picture of exactly that number of pebbles. In fact you could dispense with the picture entirely and keep only the sign, and you would successfully be thinking about that number, purely by employing its sign in your thoughts. Note that you can do operations on that sign, such as subtracting another number by manipulating the two signs via the method you learned as a child. So you have mastered (some of) the relevant math, so the sign, as you employ it, really does refer to that number.
Well I agree that I can think just with verbal signs, so long as the verbal sentences or symbolic statements mean something to me (could potentially pay rent*) or the symbols are eventually converted into some other representation that means something to me.
I can think with the infinity symbol, which doesn't mean anything to me (unless it means what I first said above: in short, "way big enough"), and then later convert the result back into symbols that do mean something to me. So I'm fine with using infinity in math, as long as it's just a form...
[edit: sorry, the formatting of links and italics in this is all screwy. I've tried editing both the rich-text and the HTML and either way it looks ok while i'm editing it but the formatted terms either come out with no surrounding spaces or two surrounding spaces]
In the latest Rationality Quotes thread, CronoDAS quoted Paul Graham: