People wouldn't have a problem with an academic journal that they believed charged a reasonable fee. But consider a typical journal published by Springer-Verlag or Mary Thomas Liebert:

• Subscription fee: $100 per issue, $1200/year
• Cost per article for non-subscribers: $30
• Fee paid by the author of each article: about $3000
• Advertising fee: I'd guess at least $5,000/page, based on the fact that each full-color page in your article usually adds $1000-$2000 to its cost
• Salary paid to reviewers: $0
• Cost of electronic publishing: about $100 per issue, divided among all subscribers

They have a problem with having a small number of subscribers. But many hobbyist groups manage to publish quality journals to equally-small audiences at a cost of under $10/issue.

The fact that people aren't jumping in to compete with lower-costs journals makes me suspect that it isn't that easy. But it's still not at all obvious why academic journals cost so much.

(The big ones, Science and Nature, are relatively inexpensive.))