I agree with all this. For the purposes of this podcast and many other articles, I have focused on dissolving confusions that are caused by a misunderstanding of what symbols are for and how they work. I could write a great deal about the advantages of symbols and what our use of symbols can tell us about ourselves... it's just not a subject I've focused on in my narrow fields of interest.
This post is a bit of shameless self-promotion, but also a pointer to an example of Yudkowskian philosophy at work that LWers may enjoy, this time concerning philosophical theories of desire.
Episode 14 of my podcast with Alonzo Fyfe, Morality in the Real World, begins to dissolve some common philosophical debates about the nature of desire by replacing the symbol with the substance, etc. Transcript and links here, mp3 here. The episode can also probably serve as a big hint of where I'm going with my metaethics sequence.
Warning: Alonzo and I are not voice actors, and my sound engineering cannot compare to that of Radiolab.