Friendship is Optimal has launched and is being published in chunks on FIMFiction.
Friendship is Optimal is a story about an optimizer written to "satisfy human values through friendship and ponies." I would like to thank everyone on LessWrong who came out and helped edit it. Friendship is Optimal wouldn't be what is today without your help.
Thank you.
Teaser description:
Hanna, the CEO of Hofvarpnir Studios, just won the contract to write the official My Little Pony MMO. Hanna has built an A.I. Princess Celestia and given her one basic drive: to satisfy everybody's values through friendship and ponies. And Princess Celestia will follow those instructions to the letter...even if you don't want her to.
Here is the schedule for the next chapters:
Friday (Nov. 16th): Chapter 4 - 5
Monday (Nov. 19th): Chapter 6 - 7
Thursday (Nov. 22th): Chapter 8 - 9
Sunday (Nov. 25th): Chapter 10 - 11, Author's Afterword
I wasn't involved in the editing process, so I don't know if you address this in future chapters, but it seems as though most people who are commenting on Equestria Daily are rejecting the concept of uploading because it would be "like dying" (similar to the Star Trek transporter thought experiment). I hope that this fear is addressed comprehensively because it seems like a major deterrence for people who would otherwise be supportive of the idea.
Also, I think that the people who "know how the story ends" because they've read about robot apocalypse analogues before are entirely missing the point, and I hope that gets expressed in the story somewhere as well.
And (as a history and archiving dork), I hope that people don't forget what came before the utopia. To lose that much human history would be devastating for me. Although I'm not sure if it goes against Celestia's programming -- people could know about the time before ponies and still have their values maximized through friendship and ponies.
(I haven't submitted it to EQD, though I plan to once I've posted all the chapters. I'm going to assume you mean FIMFiction.)
It's a bit frustrating because, to me, this came out of nowhere. The idea that "it wouldn't be you" isn't something I thought would be a problem, probably because I believe the exact opposite and assumed everyone else would. After reading so much LW, materialism and a computational theory of mind are just background assumptions.
So: ooops.
I started out thinking about making an edit to Lars and Celestia's conversation where h... (read more)