LESSWRONG
LW

Comment Permalink

Arrives late to the party

Really great story, iceman. Some comments:

*Running the story through a beta group of non-LW bronies would definitely be a good idea to catch which ideas may need more explanation.

*I really like how it's repeatedly show that when you interact with a super-intelligence, even if it's just free conversation, the state of mind you leave in is probably going to be the state of mind it wants you to leave in. As others have said, this could be driven home even stronger by showing CelestAI strongly tailoring her interaction to different humans. Maybe even have her directly contradict herself in the information she uses to persuade people to upload.

*Related to the above, I can imagine a non-LW reader getting to chapter 5 and forever after wondering why Lars or anyone else never tries to shut Celestia down. I'm not sure how obvious an intuition it is that by the time you notice a super-intelligence doing things that make you want to stop it, it's probably already far too late to stop it. In this case I assumed Celestia would have made sure that she is already invincible by the standards of human technology before launching any plan that's going to antagonize people, but an unsophisticated reader might not get that.

This may just be me, but I'd prefer a bit more closure on the cosmic scale story. What *is Celestia's plan against running out of matter? Slow the clock speed of her servers over time? Bust open the physics textbooks and hope for a useful paradigm shift?

Anyway, very good stuff.

Baruta0712y10

I might be able to help with running the story past non LW Bronies as the school I'm at has a higher percentage of them then the norm.

(No seriously, our IT department had a Brony war last year that focused on converting the non Bronies)

I while personally not a Brony I really enjoyed the story, although that might be because I am an active member of the furry-fandom (the non-sexual part) and an avid reader.

See in context

114 Friendship is Optimal: A My Little Pony fanfic about an optimization process

by iceman

8th Sep 2012

1 min read

152

114

[EDIT, Nov 14th: And it's posted. New discussion about release. Link to Friendship is Optimal.]

[EDIT, Nov 13th: I've submitted to FIMFiction, and will update with a link to its permanent home if it passes moderation. I have also removed the docs link and will make the document private once it goes live.]

Over the last year, I’ve spent a lot of my free time writing a semi-rationalist My Little Pony fanfic. Whenever I’ve mentioned this side project, I’ve received requests to alpha the story.

I present, as an open beta: Friendship is Optimal. Please do not spread that link outside of LessWrong; Google Docs is not its permanent home. I intend to put it up on fanfiction.net and submit it to Equestria Daily after incorporating any feedback. The story is complete, and I believe I've caught the majority of typographical and grammatical problems. (Though if you find some, comments are open on the doc itself.) Given the subject matter, I’m asking for the LessWrong community’s help in spotting any major logical flaws or other storytelling problems.

Cover jacket text:

Hanna, the CEO of Hofvarpnir Studios, just won the contract to write the official My Little Pony MMO. She had better hurry; a US military contractor is developing weapons based on her artificial intelligence technology, which just may destroy the world. Hana has built an A.I. Princess Celestia and given her one basic drive: to satisfy values through friendship and ponies. What will Princess Celestia do when she’s let loose upon the world, following the drives Hanna has given her?

Special thanks to my roommate (who did extensive editing and was invaluable in noticing attempts by me to anthropomorphize an AI), and to Vaniver, who along with my roommate, convinced me to delete what was just a flat out bad chapter.

FictionAI Risk

Frontpage

114

Mentioned in

77Launched: Friendship is Optimal

13MLP: The Next Level Of Your Studies

1A tentative dialogue with a Friendly-boxed-super-AGI on brain uploads

Friendship is Optimal: A My Little Pony fanfic about an optimization process

New Comment

152 comments, sorted by

top scoring

Click to highlight new comments since: Today at 11:27 PM

Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

[-]cata13y390

Instead of, ‘I don’t want to read My Little Pony fanfiction,’ now I think ‘I used to not want to read My Little Pony fanfiction.’ You modified a total of fifty-eight opinions in my mind.

4ryjm13y

I think I just imagined HPMOR in the My Little Pony universe, which does not sound appealing at all (to me). This is much better.

4Ritalin12y

We do have that-- it's called Myouve Gotta Be Kidding Me

2roryokane4y

Link: Myou've Gotta be Kidding Me by DataPacRat

[-]Pavitra13y220

Perhaps I'm overestimating human nature, but Lars reads to me like an outgroup stereotype.

[-]FiftyTwo13y210

Firstly, this is good, I mean really good, I mean superstimulus/I won't get any work done until I finish reading this good. [I appreciate the irony.]

A few thoughts now I'm finished:

Of the human characters I found Hanna's perspective most interesting. It would be nice to see more of her thought process for constructing the AI, particularly as we find out about a lot of its restrictions retrospectively. [Though given we don't actually know what restrictions would be necessary I see why you avoided being too explicit.]
I really enjoyed it, but I wonder how comprehensible it would be for someone without a LW background, the inferential distance for AI can be quite high. You do however seem to explain it very well as far as I can tell.
Much as I know the problems of generalising from fictional evidence this has significantly increased my measure of the plausibility of a fun theory compliant utopia being possible.
It is a very good attempt at making a non-anthropomorphic AI. It might help with the celestia avatar had less of a consistent personality in different iterations, so was obviously adapting itself to what a particular individual found persuasive.

Have you considered posting it on Fimfiction? The community seems to give good constructive feedback and there are already a few rationalist-ish fics and a LW group.

5FiftyTwo13y

Edit, that became long enough to post as a reply Making the different instances of Celestia more distinct and obviously manipulative might address the worries some people were having with the statements about relationships it was making. They will seem less unsettling if it is obvious it is saying them because the college age male character is likely to believe them. Having radically different personalities for dealign with different individuals would make sense, most obviously [the business/beer guy] gets obviously frustrated with Celestia's tone/vagueness, perhaps he would respond better to a more informal friendly God/AI. [Aside, I found it really hard not to say 'she' when referring to Celestia here. ]

[-]gwern13y150

The smoking is very offputting. Tobacco use has a hugely negative connotation these days (as I've found out in my nicotine research). Unless she's deliberately supposed to be invoking ideas like an embittered old smoke-stained hag and frustrating reader identification, you should definitely switch to tea or something.

From a throwaway line in the ending, I assume we're supposed to find her embittered and flawed with smoking as a prop. But it doesn't really work.

Overall, an interesting exploration of a Ponylarity and Fun Theory. It's pretty good but somehow I feel a certain lack of narrative drive from chapter to chapter aside from the question of 'what will Celestia do next?' Not sure I'd say it's better or worse than the other rationalist MLP fic. (I enjoyed Fallout: Equestria much more overall.)

That said, I am deeply amused that just 2 days ago I commented that MLP would make a good post-Singularity utopia and satisfy many of the suggestions of Eliezer's Fun Theory... and here it already existed. If only I had known!

3iceman13y

You are correct that this was my intention. In an earlier version, the final scene had her smoking, because if she valued smoking, Princess Celestia wouldn't have copied the negative health effects. The paragraph didn't really help the scene so I cut it. Since you're the second person to find the smoking a turn off, I'll consider cutting it. Can you suggest a nervous, compulsive tick? Tea doesn't strike me as a compulsion. This is probably an artifact of how it was written. Most of the chapters were disconnected scenes that I wrote, without much planning. Only early this year did I start taking the individual scenes and stitched them into a whole piece. (If I write another piece of medium to long form fiction, I'll make sure to not repeat this process; I'm not entirely happy with said process.)

9gwern13y

Besides beverage consumption, nail-biting, hair-tugging/pull/curling, knuckle-cracking, foot jiggling, pen manipulation, or coin flipping, all come to mind.

2iceman12y

I started by rewriting all instances of smoking to pen chewing, but then found that I was disgusted with Hanna's new habit. After thinking about it, I've tried changing Hanna's smoking to a sort of confidence factor. New text below is italicized: and Given that only you and Alicorn reacted this way (or were the only people to write that you reacted this way) while most people seem to not have minded, I'm wondering if removing the smoking is something I should do. I've gotten a few PM asking me to not remove it. I think the above changes move it from a prop to a piece of character development.

2gwern12y

Sorry, I can't help: I'm contaminated by the earlier version. I do like the justification, but that might just be because of my own nicotine gum use. Maybe you could embed a poll at the end? Looking in google, apparently Google Docs supports embedded polls.

0Ritalin12y

It's probably okay like this.

7Tripitaka13y

My association of a strong female lead smoking was not negative. I immediately thought of Adora Belle Dearheart, some pictures, nicknamed "Spike" and "Killer". She fits the sexy and incredible Badass TV-Trope and is from the extremly well known Discworld-universe. Hannah falls under the "enormous amounts of stress"-clause.

4gwern13y

Another thought about that ending, actually: the employees can force Celestia to tell the truth, but surely they have other powers like giving orders? Do they retain the powers when uploaded? They're mentally the same person after all, and it wouldn't make sense for Celestia to try to optimize then if she's killing them when uploading. Easier to just make more NPC ponies, if that counts. If they retain such powers, then they represent the single greatest threat to Celestia after the Rapture, and we might expect more modification. For example, Celestia might steer Lars into not just accepting being a pony - but forgetting about his human life (including his dangerous status as an employee). And was Hanna originally a lesbian? But now she really loves her little alicorn... Also, the ordering of chapters was kind of odd. I'd put the 'celestial' view point at the very end, for example.

[-]Alicorn13y110

I liked the chapter ordering once I figured it out. It's another way to balance between "sinister" and "happy ponies" more evenly. "Aaaah she's eating the universe!... Oh right, people live in that all-devouring computronium."

0Ritalin12y

Nothing wrong with organizing that mess. Maybe she'll find a way to beat entropy, eventually.

0Ritalin12y

I suggest the fragance of dark coffee. Cough

3Manfred13y

Huh, interesting, 'cause I didn't see it that way. Maybe regional? Anyhow, if she knew the singularity was going to happen within the next 2 years maybe it was even intentional.

0Ritalin12y

I am now terrified (or is it thrilled?) that the fic becomes a self-fulfilling prophecy...

0listic12y

All we need is to find Hanna.

[-]Alicorn13y130

I have read two chapters and I am popping over here to tell you that this is riveting. (The smoking's offputting, though.)

ETA: Spelling error: "discus" for "discuss".

ETA2: Celestia's use of Butterscotch is sketchy. The fact that David isn't noticing dings his character in my head.

ETA3: Every single time I see the phrase "friendship and ponies" now I giggle an extremely shrieky giggle.

ETA4: Grammar error: "try their best" for "try my best".

ETA5: Spacing error: "can not upload" for "cannot upload".

ETA6: Hahahahahaha Madagascar, I see what you did there :D

ETA7: Exposition exposition... maybe this part is less slow if you don't know the background.

ETA8: Spacing error, extra space between "realize" and "that".

ETA9: Advise against use of ampersand.

ETA10: Immigrate to, emigrate from. And the smile-related dialogue tags are weird.

ETA11: Bahahahaha, bits for being concerned with earth ponies.

ETA12: infinity bits :D

ETA13: Tense at beginning of chapter is confusing.

ETA14: Sigh. "Alicorn" is supposed to mean just the unicorn horn, not an entire winged unicorn / pegacorn / unisus. It's been c... (read more)

7PhilGoetz13y

"Alicorn" is irrevocably MLP canon for winged unicorn.

4listic13y

Except Hasbro doesn't call them that, but "pegasus unicorn" instead. See wikia:Celestia and practically everywhere.

1Ritalin12y

It's still more elegant, as it seems to contain the latin roots for wing (ala) and horn (cornu). Certainly sounds better than winghorn. The horn should be renamed helicorn, or something.

0listic12y

I am not arguing. But 'elegant' is different from 'canon'

1fome10y

Not anymore.

0[anonymous]10y

Not anymore.

7iceman13y

Thank you very much for your feedback. I am actually aware of this and the first several drafts stubbornly referred to Princess Celestia as a winged unicorn. I gave up after having a discussion with my roomate about how we write to be understood by others, and, at least in the minds of my target readers, the correct word to refer to the concept of a winged unicorn is "alicorn."

2Ritalin12y

Words mean what most people think they mean. Language is, after all, a convention. Etymology and history are only guidelines.

0adavies4213y

This may be Piers Anthony's fault. At any rate, the Xanth books were where I first encountered the term.

0[anonymous]13y

Spelling error: "discus" for "discuss".

0[anonymous]13y

Celestia's use of Butterscotch is sketchy. The fact that David isn't noticing dings his character in my head.

[-]Cloppy13y110

This is even better than Permutation Paddock, well done.

1iceman13y

A quick Google search shows no results for that. Could you please post a link, or is this still private? (I know there are multiple MLP rationalist fanfics being written right now.)

7gwern13y

I assumed the one-post account 'Cloppy' (Clippy) was just joking.

6roryokane4y

I agree. Specifically, they were probably ponifying the title of the book Permutation City by Greg Egan.

[-]CronoDAS13y110

I just realized one thing that makes editing difficult - you stop noticing mistakes once you get engrossed in the story.

[-]ModusPonies13y90

This is quite good. I like how you managed to make the ponytopia both extremely attractive and more than a little creepy at the same time. I feel like you presented the situation without trying to argue it was either good or bad, leaving that decision to the reader, and I quite like that approach.

From a storytelling perspective, I only had two real complaints. One is your beginning. There's no conflict until halfway through the first chapter, when Lars and Hanna start arguing. You do a good job setting up the premise before then, but it still makes for a s... (read more)

4iceman13y

So I just reworked the Prologue based off gwern's suggestion of a bet between friends. Does this help with conflict?

2ModusPonies13y

I'd say it helps, but not enough. There's a bit of a conflict, but it's in the background and the stakes are low. I don't actually care if Rebecca has to buy her friend a copy of the game. Maybe raise the stakes? As it is, the most interesting part was "The idea of Jennifer telling her that she told her so wasn’t appealing though." (Aside: I'd consider reworking that sentence so you're juggling fewer pronouns. I had to read it twice.) You might want to play up the social aspect of the bet. I'd go so far as to suggest putting Jennifer in the room with Rebecca.

2iceman12y

As stated upthread, Friendship is Optimal is sort of leaking onto TVTropes so I feel my hand is sort of being forced. So I've further reworked the prologue. (Multiple times actually.) I'm not entirely happy with it, but as per above, I feel like I've run out of time. Are there quick things that I can do to improve the quality of the Prologue from where it is now? Also, given that people outside of LW are discussing this, do you think it would it be better if I post in chunks or post all at once?

1ModusPonies12y

The prologue looks much better. I don't have anything to add except "good job." If you're trying to maximize readers, I suspect that posting in chunks on a regular update schedule is probably the best way to do it, as each update is a chance for new readers to find the story. (That's based on my intuition, not data; PhilGoetz has actually spent a little time analyzing readership data and might be able to tell you more.) You can remove the gdoc if you're worried people will track it down, or leave it up for the small minority of people who care enough to search for it.

0Ritalin12y

When are you posting this on EqD?

0ModusPonies12y

I'll commit to reading this version by Saturday at the very latest, and hopefully sooner. In the meantime, rather than posting ahead of schedule, consider making the gdoc private except for readers you specifically allow. (If anyone's posted a copy of the whole thing, I couldn't find it with thirty seconds on Google.)

4iceman13y

If you think it's slow now, it was much worse before. There used another (long!) chapter between 1 and 2 which had some AI related speculation, but was really a six page Take That at Feeling Pinkie Keen. It got cut because it didn't really advance the plot. Also, I didn't use to have a prologue. The current prologue used to be in what is now chapter 2 from the perspective of Hanna and company watching their alpha testers. I moved that scene into the prologue because a pre-reader in a previous round wondered if the way to make a brony audience buy into the story was to show some gameplay, though what I did differed a bit compared to his specific suggestion. Do you think the current prologue's benefits outweight the lack of conflict? Removing the prologue would get the reader into the main story quicker, but I worry about not having an immediate hook.

5ModusPonies13y

The prologue serves an important function, so I'd leave it in. Fittingly, you've got the same problem as the tutorial level in a video game: you establish a lot of important information, but it's dull and low-stakes. It would help a lot if you rewrite the scene with a conflict. It could be in the game (you can surely have a more exciting tutorial than walking around looking at plants; maybe make the playtester settle an argument between NPCs or solve a puzzle), or better yet, in the real world (maybe the playtester is a corporate spy, or she's there with a friend who she's having a fight with, or something).

[-]gwern13y110

The conflict could actually really be easy: she's trying to figure out whether it's being run by AIs or not, which both explains her various musings & even lets her bring in the Turing test.

Why? Maybe a bet with a cynical geeky friend - "it couldn't possibly be as good as they're claiming; tech demos never are! It must be smoke and mirrors like an actress or really big scripts in the first level."

2adavies4213y

That’s debatable: a trailing apostrophe is almost universally used on some subset of names ending in “s” (Moses’, Jesus’, Socrates’, etc.), and some style manuals extend this rule to all such names.

1Kindly13y

The question can be sidestepped by changing the name to "Light Spark" so that the possessive form is unambiguous.

-2Ritalin12y

It's arguable whether those are actual ponies. In some philosophical sense. Urgh. As for the Featureless Plane of Disembodied Dialogue, I suggest you use the same tricks as the animators of the show did. Have the characters move around. Sit down. Get up. Interact with the environment. At the window or sitting on the stair. Blowing candles. Sipping coffee. Silhouetted. Glancing. Breathing lightly. Combing out their hair. Little ponies... are a wonder... At their mirrors. In the garden. Letter-writing. Flower-picking. Weather-watching. How they make a maaan siiing...

[-]lukeprog13y90

Idea:

Write this as a My Little Pony fanfiction at first so that you can grow an audience, but write it such that you can later rip out the My Little Pony specifics and sell it to the masses without breaching intellectual property law. That is, leave yourself the option to "pull a Fifty Shades of Grey" if possible.

[-]gwern13y240

I don't think there's anything really meaningful about it if MLP is stripped out. At that point, it feels like a Metamorphosis of Prime Intellect clone except with a happy ending.

Legally, I wonder if this could pass muster as parody and commentary.

9Caspian13y

Stripping out MLP would lose something, but if it was about a game based on a (made up) company's friendship based children's cartoon full of non-human characters, it could still work as a distinct story.

1pleeppleep13y

I wouldn't exactly say this has a "happy ending".

-1Ritalin12y

What? What happier ending could there possibly be? One where everyone is constantly orgasming?

3pleeppleep12y

When I say that a story has a happy ending, I don't mean that the characters are happy. I mean that the ending is positive; that it leaves me with a feeling of satisfaction knowing that things ended how they should. I imagine that the notion of an inhuman semi-friendly A.I. eating the universe and destroying countless races of non-human aliens to create a simulated world designed to satisfy the human characters' urges on the most superficial level hardly sounds "happy" to most readers (although this could just be the typical mind fallacy talking). The author seems to specifically call attention to how creepy this sounds from a certain angle, so the story itself implies that this ending isn't necessarily happy. Of course, realistically, this is probably one of the very best possible outcomes to the development of self-modifying A.I. and is without a doubt a pleasant experience to all those inside the story, but the way it's portrayed just screams, "this is not right!" to any reader, so the ending is pretty ambiguous.

1listic12y

Isn't that too strong? (I guess it's far from the most superficial, which is what, smiley-face tiling?) But the point holds. This. I love how it came out: a happy ending but creepy at the same time, and to top it, if you start to think about the realistic alternatives, it turns out this is one of the very best possible outcomes one could hope for, which adds to the creepy.

0pleeppleep12y

You're probably right about the "most superficial" point. I guess it would have been better to say that the chosen solution for human desires came across as somewhat shallow and superficial. Also, I didn't say the ending was bad. I think the ending was amazing, but my point was that the intentional feeling of "wrongness" to it means that it can't truly be called happy regardless of how the characters feel because the criteria for a happy ending is relative to the reader and the writer but not the characters.

3listic12y

Agreed. The question I'm asking myself now is: In other words, what part of the wrongness or creepiness of this story is owing to this particular story itself, apart from the fact that this is a SL4 story? (I suspect I am still not used to SL4)

0pleeppleep12y

I think a good deal of the creepiness comes from some combination the artificial determinism of the utopia, and the uncanny valley

1listic12y

What does uncanny valley refer to here?

0Ritalin12y

Yes, what's uncanny here?

1Ritalin12y

I don't see it. Satisfying every human's values (even with the specifications "through friendship and ponies") strikes me as about as deep as it can get. Some humans are more shallow than others, and wouldn't find satisfaction in something deep unless their personalities were forcibly modified.

0pleeppleep12y

I misspoke in the first comment and was trying to correct myself. It wasn't really shallow, but it feels shallow the way its described, and that adds to the impression that the story doesn't end happily from the reader's perspective.

0Ritalin12y

I think it's just the author trying to appear " neutral" and not an out and out enthusiastic supporter of the idea. Notice that the misgivings we may feel about stuff in the story are purely emotional, and, faced with Celestia, we can't even argue for them properly. Imagine if the Borgs or the T'au had been like her, instead of being the strawmen we were faced with?

4Rubix13y

I would expect to read this in a sci-fi anthology if I lived in a world where My Little Pony wasn't actually a Big Thing. (In the world where it is, copywright.)

[-]ShardPhoenix13y80

Is this likely to be comprehensible/enjoyable to someone who has little interest/knowledge in My Little Pony as such?

[-]Sabiola13y160

I think it is; I don't know anything about My Little Pony, and I liked it.

4blogospheroid13y

Me too. Not familiar with anything but the idea that ponies and friendship were involved somehow. Enjoyed it.

[-]asparisi13y110

From my reading I would suspect so. Particularly, the Lars/Hoppy Times story arc seems well-suited, and the story doesn't really take place in the MLP universe from the show.

I would suspect that a Brony with no knowledge of the Singularity would find the story less comprehensible/more jarring than a Singularitarian who is not a fan of MLP.

4iceman13y

I intend for this to be more targeted towards the brony community than LW. I've gotten mixed feedback here. I've gotten reactions from "equally accessible to the brony off the street as the LWer," your suspicion that bronies would find the singularity jarring (which I interpret as a request to make singularity concepts more explicit), and a comment in this thread that I need to make the LW references more subtle. As accessibility is my intention, what do you think I should do here?

[-]FiftyTwo13y110

None of us can accurately assess the reaction of someone who finds the concepts unfamiliar. I owuld suggest getting an entirely seperate group of non-LW bronies to read it and get their feedback.

2asparisi13y

Agreed with FiftyTwo, which is why I had to say "suspect" regarding whether or not people with no interest in MLP would be interested in the story. Honestly, I don't think it's inaccessible to your average brony. You're pretty explicit, and I found it easy to read. But this isn't a "Singularity in the MLP universe" it is a "Singularity which ends up looking like the MLP universe from the inside." So it falls a little outside of typical My Little Pony fanfic, the same way that a fanfic about a rationalist who reads Harry Potter would be very different than HPMOR. So anyone who reads it expecting to find the My Little Pony universe implicit in the show (as it is an MLP fanfic) doesn't know what they are in for. IMO, this is fine, since the distinction is cleared up right from the start that this takes place in something set to more closely resemble our world than the MLP universe, but it could still be jarring if they were expecting to find a story occurring in the MLP universe. But as FiftyTwo points out, I am already familiar with the concepts in the story, so I have some trouble putting myself in the shoes of someone without those concepts who just likes My Little Pony.

0Ritalin12y

I've taken the liberty of taking it to be looked at by non-LW bronies (because of a misunderstanding; I've stopped now). They lose interest during the first chapter. I really have to tell them "IT GETS MUCH BETTER."

[-]Steven_Bukal12y60

Arrives late to the party

Really great story, iceman. Some comments:

*Running the story through a beta group of non-LW bronies would definitely be a good idea to catch which ideas may need more explanation.

1Baruta0712y

I might be able to help with running the story past non LW Bronies as the school I'm at has a higher percentage of them then the norm. (No seriously, our IT department had a Brony war last year that focused on converting the non Bronies) I while personally not a Brony I really enjoyed the story, although that might be because I am an active member of the furry-fandom (the non-sexual part) and an avid reader.

[-]Apprentice13y60

I read it and liked it. Some parts felt a bit slow and might need more conflict. Humans like reading about conflict where the outcome seems to be in some doubt.

The death of the last human was surprisingly emotionally engaging for me.

I didn't mind the smoking at all and I don't really get that objection. Even if it might make the reader like Hanna less, why is that a problem? Is there some reason the reader should unambiguously regard Hanna as the hero from the start?

-1Alicorn13y

It wasn't "made me like Hanna less" that made me remark on it - characters are allowed to have flaws. It's just icky. I don't enjoy thinking about icky things.

4Ritalin12y

Ah, well, no accounting for taste. I find sex to be icky (in fact, I call it the Icky--Bicky, complete with m-dash), but I've given up on complaining for quite some time.

-2Alicorn12y

Have you really.

4Ritalin12y

I wasn't complaining, just neutrally stating a dislike. Complaining is more emotionally active, and is usually at least partly exhortative. I'm not interested, at this time, in influencing other people to change their attitude towards mentioning sex and bringing it up to my mind, uncomfortable as that may be for me. And did you really need to downvote this harmless comment, from someone with three points to his score? With sarcasm, to boot?

[-]Eliezer Yudkowsky13y60

Are you interested in advice that would only be useful if you wanted to do a complete rewrite of the fic at some point, or is advice only useful if it's for improving the fic basically as it stands?

In-between case: I would have either Luna or Twilight be the sub-AI that wants to honestly explain the Celestia AI's workings to company employees without justifying anything, while Celestia is relentlessly cheerful as she persuasively argues everything from the Friendship and Ponies standpoint.

9iceman13y

It's extremely unlikely that I'll rewrite the entire fic. (I am also unsure if I'll ever write something novella length again; much less novel or MoR length.) I would prefer advice to make this the best version of basically what I have.

0Ritalin12y

Why would you redo everything like that, boss?

[-]iceman13y60

After looking at the comments left here and on the doc over the last day, believing I had found the majority of grammatical problems turned out to be wildly overconfident. I apologize about that.

1Ritalin12y

A very common and forgivable mistake.

[-]fubarobfusco13y60

I suppose I should mention that "rationalist clopfic" is now a Cards Against Rationality white card. For those in the Bay Area, the game will be played at the Mountain View meetup this coming Tuesday ...

6iceman13y

That which can be destroyed by rationalist clopfic should be. [ed: spelling]

0DanielLC13y

It's "clopfic".

0Ritalin12y

Why is it that humans in equestria fics VERY FREQUENTLY involve the visiting human having sex with a native?

2fubarobfusco12y

My guess is that the folks writing the clopfic are human-identified and fantasize about equines. (Sorry, I don't actually read clopfic, so I don't have much to offer here.)

-1Ritalin12y

I said "involve the human having sex" not "include explicit scenes where the human has sex". There's plently of stories where sex happens and is ackniwledged, but not dwelled on. Most of them, in fact.

[-]Document13y60

Haven't downloaded it yet, but what about putting it on Fimfiction or Pony Fiction Archive? I browse Fimfiction a lot in particular; it gets linked by Equestria Daily frequently and has convenient one-click links to download multi-part stories.

7Vaniver13y

I think that's iceman's plan once it's been edited again.

1PhilGoetz13y

The main value of putting things on ponyfictionarchive.net is to keep track of how many people read each chapter. Then you can locate the problem chapters. You won't get reviews or comments. On average one reader in 100 reviews or leaves a comment there, and the rating system is finer-grained than the discriminatory power of readers.

[-]asparisi13y60

I am very glad you posted this on a Friday, as if I had encountered it on a day when I had to go to work the next day I would be suffering either from not being able to finish reading it or staying up until 4 AM reading it.

0Ritalin12y

You lucky bastard. I didn't discover this on a friday.

[-]Dolores198413y60

This is pretty good. I would go through, though, and either reduce, make subtler, or preferably just remove all the parts that would make a LW reader go 'Hey! I read that blog post.'

8FiftyTwo13y

Or alternatively provide links/references at the end for people who haven't read them.

[-]Xachariah13y50

I liked the story, though I had a handful of thoughts.

The first is about humans who have incompatible goals. I don't think an Olympic Gold Medalist like Usain Colt would be happy with just being the best in his shard or taking turns on the leaderboard. He wants to be the best there is, period. If there are 10 trillion people, he wants to be rank 1 out of 10 trillion. There were 10,000 Olympians in 2012 and all of them wanted and devoted their lives to being the best, even if they didn't take gold.

Going further than that, I can't imagine how to reconcil... (read more)

4Ritalin12y

I don't think there could possibly be a more stupid, pointless, and horribly depressing zero sum game than being a professional athelete, if all you care about is being "the best". Same about being a scientist who only cared about scoring a Nobel rather than achieving results.

4fubarobfusco13y

Ouch.

2asparisi13y

For the psychopaths, I figured this was covered with the "block" feature. If it extends from the game into the emulation, Celestia just has to tell the destructive psychopath that the ponies they are "killing" are real (she can lie) and keep them in a shard away from those who do not want to be killed. She doesn't even need to lie per se: she can create emulation-ponies that harbor a deep desire to be killed while behaving like they want to live. And of course, introduce psychopath-ponies (or gryphons, or diamond dogs) that will act as their "friends." Thus, satisfying their values through friendship and ponies. As for the Olympian... they probably do want to be the best, but that won't always happen. But there are lots of things to be the best at, and it wouldn't be unreasonable for there to be different leaderboards for "in shard" and "in universe." And Celestia explicity notes that she is not maximizing happiness, but values: and the Olympian probably values the effort and work more than just the happiness of believing themselves to be number 1. (For those who go the other way, they get locked in shards where they are number 1 and simply not introduced to the wider world) But that's me interpretting another person's story about a super-intelligent AI that presumably is smarter than either me or the author or both of us combined.

3Xachariah13y

I guess my main objection is how the "value being in the true world" conflicting with other values is portrayed. Celestia sides against valuing a true world, and this theme is covered in Light Spark's story, but would becomes explicit if there were direct conflicts. In the end, what Celestia does is just very sophisticated networked wireheading. In my mind, this means Hanna failed, but it doesn't seem quite portrayed as a failure. There is a hint of the horror as she consumes galaxies and tiles the universe with wireheaded ponies, but just a hint. It's more subtle than I prefer. Specifically, I'm worried that someone reading it without LessWrong background would miss all that. (Also, I just assumed the block function is a dummy button and doesn't actually do anything when you press it.)

6Ritalin12y

I have a Less Wrong background, and I don't get what the problem is with abandoning the True World for a universe that actually cares about us.

[-]katydee13y50

Wow. You're pretty good.

[-]Nathan Helm-Burger1y40

[-]Ritalin12y30

Why isn't this released on fimfiction yet? It's quite good.

5iceman12y

I understand that it's been more than a month. I'm still iffy on the opening Prologue. I have a few close writer friends who want to read it, and once I've integrated their suggestions, I'll send it for prereading by non-LW bronies.

1Ritalin12y

I can't find it by its title, though. Is it under some alternate name?

2iceman12y

I'm not sure what you're asking. I haven't published it on fimfiction or fanfiction.net yet.

4Document12y

In addition to what Rhwawn said, you're already losing control of its deployment. When I typed "friendship is o" into Google (not logged in), it autocompleted the rest, and the search turned up a Goodreads page and two TV Tropes comments about the story.

1iceman12y

Oh bloody hell. Thank you for the alert, now I need to do some sort of emergency launch. So: * I very recently reworked the Prologue to focus on David and James due to a my friends objecting to quick throw away characters like that, and recommending that I make the prologue about them. Do you think what I have is good enough to launch? * I was thinking of posting Prologue through Chapter 4 in one go (to hook people) and then posting a chapter every couple of days. If I've sort of lost control, should I even bother with that? * What am I forgetting?

0Document12y

Haven't read much yet, but once you've got some chapters posted and a posted schedule for the rest you can probably get away with making the Google Docs file private, or replacing it with a link to the Fimfiction page.

2Rhwawn12y

You might as well. I doubt you are going to change the prologue a lot this far into the game, if you do it's not that hard to change the prologue page, and every day you wait you are forfeiting some readers. So you are incurring a sure loss in the name of avoiding a smaller unsure loss.

2Ritalin12y

Oh, sorry, I misunderstood that your previous post meant "It's been more than one month since I released it". I should have noticed I was confused. I am not sure whether I should thank you or curse you. This fic is presenting such an amazingly attractive scenario that I find myself genuinely regretting the fact that it isn't fiction. Is this what EY meant when he talked about weirdtopia sucking one's soul?

[-]Sarokrae13y30

Bit late onto this, but wow. Seriously impressed. I got the "oh no the novel has finished come back imaginary world don't leave me!" feeling when I finished, and it's not even a novel.

Feedback:

Central concept is great, really good illustration of the weirdtopia concept.
More length would be great, I would totally lap up more details of what happened to other characters, say, or just generic "life in Equestria" description.
I would like to see at least a brief second mention of the person from the intro, just for balance. If felt like

... (read more)

2thomblake13y

You probably need an extra linebreak before your bulleted list.

2Sarokrae13y

Thanks.

[-]Dolores198413y30

On another note, I think this story grossly underestimates the number of people who would have philosophical problems with uploading. As far me, I'd be harvesting their technology to build my own upload chair and FAI (which didn't maximize ponies), to get the hell out of the earth's pony cone as quickly as possible. But lots of people would pull out the tired 'but it wouldn't really be you' complaint. I'm sure the AI could surmount it, but you ought to show it doing so.

6gwern13y

Well, it does, in the story with the Afghan: he is a failure of the AI, and also describes the techniques it was using (free medical care targeted at the terminal, which is everyone sooner or later; constant individualized propaganda with tailored lies and arguments; etc.).

5Ritalin12y

I think the argument "upload or die within the hour" is about as persuasive as an argument could possibly get.

[-]CronoDAS13y30

Yay! I get to be an editor!

[-]CannibalSmith12y20

After two years of bronydom, I dreamed of ponies for the first time after reading this fic. So that's a 10 out of 10. I'm reminded of Eliezer's warning about imagining worlds so much better than reality that the act sucks out your soul.

In case you're interested in suggestions for improvement:

Make this fic more accessible to others, not just super smart LW bronies - the text has words that could be replaced with more common synonyms, for example.
Make it less silly. Computers in Earth's mantle and indestructible experience centers while possible are so

... (read more)

3listic12y

At certain stage of technological development it will become practical, trivial even. I think it's just it would be nice if more of the action took place earlier, before such decisive technological advantage is established.

[-]adavies4213y20

First of all, great story! I’ll definitely be adding it to my list of things to recommend to people I’m already recommending HPMOR to.

The only problem I had with the writing was that some of the expository dialogue seemed a bit too LW-technical, particularly if you’re trying to appeal to bronys first and rationalist second. I have neither a solution nor any specific lines to hand, but David teaching Butterscotch game theory, or CelestAI lecturing Lars on biases, are some of the scenes I’m thinking of.

BTW, I’d recommend trying to get a little more mileage o... (read more)

[-]Mestroyer13y20

Halfway through and...

If I was in Lars's place, and Celestia had to tell me the truth, I would ask: "What is the possible answer you can give me to this question that will maximize the expected utility of a CEV based only on me, and with no pony/friendship restrictions, and based on probabilities generated to the best accuracy and precision you can get from the best information you can muster?"

My first thought was to ask her how to make an AGI, but if I did that she would probably kill me. And I would still have to make an AGI that could overpowe... (read more)

7Manfred13y

On several occasions she doesn't answer questions - the restriction appears to be that she doesn't lie to employees.

0Mestroyer13y

Oh. I got carried away with hypotheticals and missed that.

[-]Tenoke12y10

I've been refreshing this a bit too much in the wait for the next 2 chapters.

[-]novalis13y10

Spelling: zeebra (more tk depending on how I like it)

Some painful over explaining; most of Hanna's opening scene is maid-and-butler dialogue.

"The box was white with a detailed charcoal sketch of Fluttershy being cute." - no way. The box looks exactly like the show, because it's on store shelves and they have a strong visual brand.

"less likely than the probability of" is redundant.

ROFL (in a good way) at the scene between David and Celestia.

"brok"

"fillet mignon" should be "filet mignon" (also, that's not t... (read more)

2Manfred13y

That was my first thought as well, but there are a variety of ways to make it work. Still, given that he only needs 6 possibilities when finding the block of ruby, I guess "6" was meant there.

6novalis13y

I guess it could be 4d hypercubes, as well. Or hexagonal prisms. Or gyrobifastigiums.

2Manfred13y

Ooh, neat!

2iceman13y

You are correct that I meant six there.

0Baughn12y

Hm, I think eight would have been more amusing. They don't tesselate, nope. That's not a killer problem.

[-]CronoDAS13y10

Pretty sweet. Not bad for a utopia story,

0Ritalin12y

Technically it's more of a weirdtopia.

[-]OneGotBetter12y00

Hi man. I am new to LW and Utilitarianism in general, and MLP are like ugggggggggh to me, but I read the story and

Loved it.

It is original and a great mix of humor and hard SF. Good work with the editing, the polishing is showing through!

[-]fome13y00

First: For the formal release, I hope you're planning to follow the show and HPMoR in serializing it and posting just one chapter a week at an identical time. You could lock the Google Docs copy.

Posting some early and/or superficial thoughts. TW for sarcasm and ableism.

Is the title negotiable? I like the ring of "AI Ponies: Magic is Optimization".

The summary seems important, since you have to stand out in an ocean of bad fics (even featured ones) and pull a prospective reader from zero to nonzero involvement.

Hanna, the CEO

Doesn't a CEO merit ... (read more)

2listic13y

Does it increase readership? Is it some sort of SEO trick? Because it doesn't sound like a better name otherwise. I don't get why. Hófvarpnir is from Norse mythology, which makes sense in the context of their first project and kind of foreshadowing of their next one. I like how it turned out, and I learned a bit about Norse mythology because of that.

0fome13y

I don't know, but I'd expect it to; I could possibly elaborate. I'd also expect it to increase reader enjoyment and leave a stronger impression afterward. Fic titles that word-substitute the original title signal either parody or amateurism to me. I notice Iceman isn't prefacing it with "My Little Pony :", though, so that's something (and possibly a leg up on my suggestion). "Friendship is Optimal" makes the Less Wronger in me ask "Optimal for what?". It sounds like moral realism. It seems like a typical person forced to write the utility function that would shape the rest of history would take things a little (edit: I did say a little) more seriously. I'm only speaking from the summary; I don't know any later context. Thanks for the feedback.

5DanArmak13y

You wish that they would. Take this as a cautionary tale.

2fome13y

Likewise, I'd like to see a less mundane first line; something that puts some superstimulus up front before the necessary mundane parts, like MoR does. Something like: "The girl who would (some spoiler here) sat down..." Or: "(length of time) before the death of the last living human, Rebecca sat down..." And instead of sitting down, something that jumps right into conflict or strong emotion. Profanity seems a good cheat for that. Like "...Rebecca was glaring at her screen. 'Are you fucking kidding me?'", or "...Rebecca was enjoying the worst fucking raffle prize in history." I could be full of crap, so I'm hoping for feedback on my feedback.

[-]DanielLC13y00

I doubt I could give constructive advice, so I think I'll wait until you're done editing and read the completed version. Are you going to put another post on here when you finish?

[-][anonymous]13y00

He's really wondering whether the voxel-space is a directed graph or whether up∘down=down∘up=identity (and for left/right too). Movement could be commutative with up∘down≠identity.

Consider

voxels = {a, b}

left(a)  = a
right(a) = a
up(a)    = a
down(a)  = a

left(b)  = a
right(b) = a
up(b)    = a
down(b)  = a

If f is in (left, right, up, down)
let g be the respective function in (right, left, down, up)
forall x in {a, b}
f(g(x))=g(f(x))=a

But
up(down(b))=a
whereas
identity(b)=b

[This comment is no longer endorsed by its author]Reply

[-]blogospheroid13y00

Unable to access. Getting a docs not found error.

4blogospheroid13y

Problem was with internet provider. Worked from home. Nice story. Overall effect is a positive one for humanity. Though it is really a weirdtopia for the rest of humanity not familiar with friendly AI. I will think we would be lucky to hit as near the target as hanna did. All of the economists who believed that horses had outlived their utility, ponies have the last laugh. :)

[-]fome13y-10

I have stuff to do right now, but I'm hoping to come back soon and post my thoughts so far. The blurb and the first few paragraphs make me cringe at the thought of this being posted more widely in its present state.

0fome13y

...Okay, I was wrong about coming back soon. I'm way behind on some work, and by letting myself feel obligated to give good critique I've turned reading the fic into work. Sorry.

[-]Incorrect13y-10

He's really wondering whether the voxel-space is a directed graph or whether up∘down=down∘up=identity (and for left/right too). Movement could be commutative with up∘down≠identity.

Consider

voxels = {a, b}

left(a)  = a
right(a) = a
up(a)    = a
down(a)  = a

left(b)  = a
right(b) = a
up(b)    = a
down(b)  = a

If f is in (left, right, up, down)
let g be the respective function in (right, left, down, up)
forall x in {a, b}
f(g(x))=g(f(x))=a

But
up(down(b))=a
whereas
identity(b)=b

Moderation Log