You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

fubarobfusco comments on Friendship is Optimal: A My Little Pony fanfic about an optimization process - Less Wrong Discussion

63 Post author: iceman 08 September 2012 06:16AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (145)

You are viewing a single comment's thread. Show more comments above.

Comment author: Xachariah 08 September 2012 10:20:34PM *  4 points [-]

I liked the story, though I had a handful of thoughts.

The first is about humans who have incompatible goals. I don't think an Olympic Gold Medalist like Usain Colt would be happy with just being the best in his shard or taking turns on the leaderboard. He wants to be the best there is, period. If there are 10 trillion people, he wants to be rank 1 out of 10 trillion. There were 10,000 Olympians in 2012 and all of them wanted and devoted their lives to being the best, even if they didn't take gold.

Going further than that, I can't imagine how to reconcile the 1% of the population who are psychopaths, or how to reconcile the 1% of the 1% who are truly destructive. Ted Bundy killed because he wanted to feel his power over others and murder was the best high he had ever had; such a man would think it better to harm real people instead of constructs.

I assume that Celestia would modify them rather than make everyone mutually delusional in thinking they were the best (or killing others), but I had hoped that would be touched on. I understand mutually incompatible goals are a problem CEV and not just your story. The way AI resolves these mutually incompatible goals is a lot of what's scary about an AI controlled future.

(As a minor sidepoint, what of humans who are okay with ponies but value not living in a matrix or wireheading. Do they get engineered pony bodies or do they just get that value modified out? I'd assume the latter or that anyone who holds that value ends up like Abdul.)

The other main thought I had was that I liked chapter 10 as an ending rather than 11. It has a strong emotional resonance with me that individual stories don't. Although I do realize it would change the tone of the ending somewhat significantly. Vafgrnq bs na 'rirelobql yvirf unccvyl rire nsgre, fbegn' raqvat vg jbhyq cebonoyl or 'bu Tbq na bzavcbgrag NV vf gvyvat gur havirefr jvgu cbavrf naq rirelobql'f BX jvgu gung' raqvat.

I do want to reiterate how much I liked your story. Very well done and thank you for sharing.

Comment author: fubarobfusco 09 September 2012 12:59:01AM 3 points [-]

Usain Colt

Ouch.