Since no one really discussed it yet... by far, I found the fear of uploading to be the most gut-wrenching part. No new ideas, but putting it into a story context made it hit home.
In terms of the utopia, the most concerning part for me was that Celestia did not seem to (ROT13) qrpvqr gung uhznaf inyhr guvatf va gur rkgreany raivebazrag. Pryrfgvn jnf unccl gb yvr va beqre gb znxr crbcyr(/cbavrf) unccl, rira jura gur cbal va dhrfgvba fcrpvsvpnyyl qrfverq gb xabj jung jnf tbvat ba va gur rkgreany jbeyq. Pryrfgvn jnf unccl gb fubj gur cbal n cvpgher bs gur rkgreany jbeyq qrfvtarq gb znkvznyyl fngvfsl inyhrf, vafgrnq bs (jung V jbhyq frr nf) ernyyl fngvfslvat gur cbal'f qrfver gb xabj.
So you think you can guess that character's desire more accurately than a godlike AI with full access to her mind could?
So Eliezer said in his March 1st HPMOR progress report:
So I read that and it was certainly very much worth reading - thanks for the recommendation! Obviously, the following contains spoilers.
I'm confused about how the story is supposed to be "terrifying". I rarely find any fiction scary, but I suspect that this is about something else: I didn't think Failed Utopia #4-2 was "failed" either and in Three Worlds Collide, I thought the choice of the "Normal" ending made a lot more sense than choosing the "True" ending. The Optimalverse seems to me a fantastically fortunate universe, pretty much the best universe mammals could ever hope to end up in, and I honestly don't see how it is a horror novel, at all.
So, apparently there's something I'm not getting. Something that makes an individual's hard-to-define "free choice" more valuable than her much-easier-to-define happiness. Something like a paranoid schizophrenic's right not to be treated,
So I'd like the dumb version please. What's terrifying about the Optimalverse?