For the psychopaths, I figured this was covered with the "block" feature. If it extends from the game into the emulation, Celestia just has to tell the destructive psychopath that the ponies they are "killing" are real (she can lie) and keep them in a shard away from those who do not want to be killed. She doesn't even need to lie per se: she can create emulation-ponies that harbor a deep desire to be killed while behaving like they want to live. And of course, introduce psychopath-ponies (or gryphons, or diamond dogs) that will act as their "friends."
Thus, satisfying their values through friendship and ponies.
As for the Olympian... they probably do want to be the best, but that won't always happen. But there are lots of things to be the best at, and it wouldn't be unreasonable for there to be different leaderboards for "in shard" and "in universe." And Celestia explicity notes that she is not maximizing happiness, but values: and the Olympian probably values the effort and work more than just the happiness of believing themselves to be number 1. (For those who go the other way, they get locked in shards where they are number 1 and simply not introduced to the wider world)
But that's me interpretting another person's story about a super-intelligent AI that presumably is smarter than either me or the author or both of us combined.
I guess my main objection is how the "value being in the true world" conflicting with other values is portrayed. Celestia sides against valuing a true world, and this theme is covered in Light Spark's story, but would becomes explicit if there were direct conflicts. In the end, what Celestia does is just very sophisticated networked wireheading.
In my mind, this means Hanna failed, but it doesn't seem quite portrayed as a failure. There is a hint of the horror as she consumes galaxies and tiles the universe with wireheaded ponies, but just a ...
[EDIT, Nov 14th: And it's posted. New discussion about release. Link to Friendship is Optimal.]
[EDIT, Nov 13th: I've submitted to FIMFiction, and will update with a link to its permanent home if it passes moderation. I have also removed the docs link and will make the document private once it goes live.]
Over the last year, I’ve spent a lot of my free time writing a semi-rationalist My Little Pony fanfic. Whenever I’ve mentioned this side project, I’ve received requests to alpha the story.
I present, as an open beta: Friendship is Optimal. Please do not spread that link outside of LessWrong; Google Docs is not its permanent home. I intend to put it up on fanfiction.net and submit it to Equestria Daily after incorporating any feedback. The story is complete, and I believe I've caught the majority of typographical and grammatical problems. (Though if you find some, comments are open on the doc itself.) Given the subject matter, I’m asking for the LessWrong community’s help in spotting any major logical flaws or other storytelling problems.
Cover jacket text:
Hanna, the CEO of Hofvarpnir Studios, just won the contract to write the official My Little Pony MMO. She had better hurry; a US military contractor is developing weapons based on her artificial intelligence technology, which just may destroy the world. Hana has built an A.I. Princess Celestia and given her one basic drive: to satisfy values through friendship and ponies. What will Princess Celestia do when she’s let loose upon the world, following the drives Hanna has given her?
Special thanks to my roommate (who did extensive editing and was invaluable in noticing attempts by me to anthropomorphize an AI), and to Vaniver, who along with my roommate, convinced me to delete what was just a flat out bad chapter.