This makes sense, but I'm not sure Eliezer will be that reassured with it as applied to him.

This strikes me as the right answer. Things like Star Trek and Tolkien are incredibly powerful for very small subsets of the population because their creators make risky aesthetic and narrative choices. It isn't so much that fans feel they must come to the defense of their preferred works, but that those works speak to them in rare and intense ways that are really distasteful to most people. So fans bask in the uncommon power of their fan-objects and disregard prevailing opinion. People aren't as fanatical about things like Indiana Jones or Animal Farm because their appeal is shallow and broad: everyone seems to agree that Indiana Jones is a sympathetic and entertaining character and Animal Farm is a clever allegory, but they only speak to one thing, and one thing that is widely understood. Star Trek, by comparison, is an immersive universe that goes down peculiar and deep paths that explore culture, power, ethics, and history among other things. It is not so much that all fan-objects possess objective awfulness, but they all do sacrifice wide appeal for a constrictive spiritual completeness.

I once read advice by a successful author, who claimed that if you want to build up a loyal fanbase, then you want to have people who absolutely hate your work, because if nobody cares enough to hate your books then they're not distinctive enough and nobody will love them either:

The best way to find your target audience is to write something original! When you’re truly original, the mainstream readers of that genre will often consider your work outrageous, or shocking, or insane, or unique, or weird, or all these things, but that’s okay. If it’s your original voice, stand proud and pick one of your books to slam down the throats of the entire obvious audience. Then be strong enough to deal with the high percentage of hate reviews you will certainly get from those who don’t “get” your work. A lot of authors can’t handle hate reviews. But a bad review simply means someone outside your target audience found your book. The angrier the review, the further removed from your target audience they are. But along with the hate reviews, you’ll get some great ones.

The reason you’ll get some great reviews for your original writing is because I don’t care what you’re selling, there’s a market for it! What I’m saying, if you’re not offending a significant number of readers, your writing is probably not very original. And the less original you’re writing, the less loyal your fan base will be. [...]

Yes, Saving Rachel was my third book, but when I wrote it, I realized it would be the key to finding my target audience, because it divided people like crazy. Most either hated it or loved it. If I had known then what I know now, Saving Rachel would’ve been my first book. But that’s not important. What’s important is that you write a unique, original book that will divide the reading world into two camps: those who love your writing and those who hate it. Those who hate it will give you angry, spiteful reviews. That’s the bad news. The good news is they’ll never buy your books again, so that will end their angry reviews!

I know what you’re thinking: “Why is alienating half the book buying audience a good thing?” The answer is it proves you’re original. And the more unique and original your writing, the deeper and more loyal your target audience will be. I mean, there’s a limit—you don’t want everyone to hate your work! Ideally, you’d hope for 60% to love your Target Book, 30% to hate it, and you’ll always have 10% who can’t decide, which means they’re probably open to trying another of your books.

Once you know your target audience you’ll write directly to them. If you don’t get a lot of bad reviews with your Target book, you’re not original enough. I’m not talking about your initial reviews. Almost all of those will be positive. I’m talking about the reviews you get after your book starts moving up significantly. That’s when the bad reviews start creeping in. But that’s a good thing because it will help you identify and grab the attention of your Target Audience.

Locke, John (2011-06-15). How I Sold 1 Million eBooks in 5 Months! Telemachus Press, LLC. Kindle Edition.

(That said, according to the Amazon reviews of that book, most of his success came from paid reviews, so might want to take what he says with a grain of salt.)

I think you are on to something. When you think about it, humans are different enough that it's hard to create a work that everyone thinks is great. You might be able to create a work that nobody profoundly dislikes, but such a work is likely to be so bland, watered-down, and lacking in risks that nobody is profoundly thrilled with it, either. Creating a work that resonates with the worldview and experience of a certain group to a high magnitude can make it inaccessible or laughable to other groups of people with different values.

There may be a "Conversation of Fandom" of some sort going on: for every enthusiastic fan you produce with a work, you must also produce someone who hates it.

Contra Bond, it's not badness that produce fandom. Rather, elements with a high variance of appeal produce both fans in some groups of people, and badness from the perspective of other groups of people. These groups can even overlap, in the case of So Bad It's Good.