Instead of, ‘I don’t want to read My Little Pony fanfiction,’ now I think ‘I used to not want to read My Little Pony fanfiction.’ You modified a total of fifty-eight opinions in my mind.
Firstly, this is good, I mean really good, I mean superstimulus/I won't get any work done until I finish reading this good. [I appreciate the irony.]
A few thoughts now I'm finished:
Of the human characters I found Hanna's perspective most interesting. It would be nice to see more of her thought process for constructing the AI, particularly as we find out about a lot of its restrictions retrospectively. [Though given we don't actually know what restrictions would be necessary I see why you avoided being too explicit.]
I really enjoyed it, but I wonder how comprehensible it would be for someone without a LW background, the inferential distance for AI can be quite high. You do however seem to explain it very well as far as I can tell.
Much as I know the problems of generalising from fictional evidence this has significantly increased my measure of the plausibility of a fun theory compliant utopia being possible.
It is a very good attempt at making a non-anthropomorphic AI. It might help with the celestia avatar had less of a consistent personality in different iterations, so was obviously adapting itself to what a particular individual found persuasive.
Have you considered posting it on Fimfiction? The community seems to give good constructive feedback and there are already a few rationalist-ish fics and a LW group.
The smoking is very offputting. Tobacco use has a hugely negative connotation these days (as I've found out in my nicotine research). Unless she's deliberately supposed to be invoking ideas like an embittered old smoke-stained hag and frustrating reader identification, you should definitely switch to tea or something.
From a throwaway line in the ending, I assume we're supposed to find her embittered and flawed with smoking as a prop. But it doesn't really work.
Overall, an interesting exploration of a Ponylarity and Fun Theory. It's pretty good but somehow I feel a certain lack of narrative drive from chapter to chapter aside from the question of 'what will Celestia do next?' Not sure I'd say it's better or worse than the other rationalist MLP fic. (I enjoyed Fallout: Equestria much more overall.)
That said, I am deeply amused that just 2 days ago I commented that MLP would make a good post-Singularity utopia and satisfy many of the suggestions of Eliezer's Fun Theory... and here it already existed. If only I had known!
I liked the chapter ordering once I figured it out. It's another way to balance between "sinister" and "happy ponies" more evenly. "Aaaah she's eating the universe!... Oh right, people live in that all-devouring computronium."
I have read two chapters and I am popping over here to tell you that this is riveting. (The smoking's offputting, though.)
ETA: Spelling error: "discus" for "discuss".
ETA2: Celestia's use of Butterscotch is sketchy. The fact that David isn't noticing dings his character in my head.
ETA3: Every single time I see the phrase "friendship and ponies" now I giggle an extremely shrieky giggle.
ETA4: Grammar error: "try their best" for "try my best".
ETA5: Spacing error: "can not upload" for "cannot upload".
ETA6: Hahahahahaha Madagascar, I see what you did there :D
ETA7: Exposition exposition... maybe this part is less slow if you don't know the background.
ETA8: Spacing error, extra space between "realize" and "that".
ETA9: Advise against use of ampersand.
ETA10: Immigrate to, emigrate from. And the smile-related dialogue tags are weird.
ETA11: Bahahahaha, bits for being concerned with earth ponies.
ETA12: infinity bits :D
ETA13: Tense at beginning of chapter is confusing.
ETA14: Sigh. "Alicorn" is supposed to mean just the unicorn horn, not an entire winged unicorn / pegacorn / unisus. It's been c...
I just realized one thing that makes editing difficult - you stop noticing mistakes once you get engrossed in the story.
This is quite good. I like how you managed to make the ponytopia both extremely attractive and more than a little creepy at the same time. I feel like you presented the situation without trying to argue it was either good or bad, leaving that decision to the reader, and I quite like that approach.
From a storytelling perspective, I only had two real complaints. One is your beginning. There's no conflict until halfway through the first chapter, when Lars and Hanna start arguing. You do a good job setting up the premise before then, but it still makes for a s...
The conflict could actually really be easy: she's trying to figure out whether it's being run by AIs or not, which both explains her various musings & even lets her bring in the Turing test.
Why? Maybe a bet with a cynical geeky friend - "it couldn't possibly be as good as they're claiming; tech demos never are! It must be smoke and mirrors like an actress or really big scripts in the first level."
Idea:
Write this as a My Little Pony fanfiction at first so that you can grow an audience, but write it such that you can later rip out the My Little Pony specifics and sell it to the masses without breaching intellectual property law. That is, leave yourself the option to "pull a Fifty Shades of Grey" if possible.
I don't think there's anything really meaningful about it if MLP is stripped out. At that point, it feels like a Metamorphosis of Prime Intellect clone except with a happy ending.
Legally, I wonder if this could pass muster as parody and commentary.
Is this likely to be comprehensible/enjoyable to someone who has little interest/knowledge in My Little Pony as such?
From my reading I would suspect so. Particularly, the Lars/Hoppy Times story arc seems well-suited, and the story doesn't really take place in the MLP universe from the show.
I would suspect that a Brony with no knowledge of the Singularity would find the story less comprehensible/more jarring than a Singularitarian who is not a fan of MLP.
None of us can accurately assess the reaction of someone who finds the concepts unfamiliar. I owuld suggest getting an entirely seperate group of non-LW bronies to read it and get their feedback.
Arrives late to the party
Really great story, iceman. Some comments:
*Running the story through a beta group of non-LW bronies would definitely be a good idea to catch which ideas may need more explanation.
*I really like how it's repeatedly show that when you interact with a super-intelligence, even if it's just free conversation, the state of mind you leave in is probably going to be the state of mind it wants you to leave in. As others have said, this could be driven home even stronger by showing CelestAI strongly tailoring her interaction to different hu...
I read it and liked it. Some parts felt a bit slow and might need more conflict. Humans like reading about conflict where the outcome seems to be in some doubt.
The death of the last human was surprisingly emotionally engaging for me.
I didn't mind the smoking at all and I don't really get that objection. Even if it might make the reader like Hanna less, why is that a problem? Is there some reason the reader should unambiguously regard Hanna as the hero from the start?
Are you interested in advice that would only be useful if you wanted to do a complete rewrite of the fic at some point, or is advice only useful if it's for improving the fic basically as it stands?
In-between case: I would have either Luna or Twilight be the sub-AI that wants to honestly explain the Celestia AI's workings to company employees without justifying anything, while Celestia is relentlessly cheerful as she persuasively argues everything from the Friendship and Ponies standpoint.
After looking at the comments left here and on the doc over the last day, believing I had found the majority of grammatical problems turned out to be wildly overconfident. I apologize about that.
I suppose I should mention that "rationalist clopfic" is now a Cards Against Rationality white card. For those in the Bay Area, the game will be played at the Mountain View meetup this coming Tuesday ...
Haven't downloaded it yet, but what about putting it on Fimfiction or Pony Fiction Archive? I browse Fimfiction a lot in particular; it gets linked by Equestria Daily frequently and has convenient one-click links to download multi-part stories.
I am very glad you posted this on a Friday, as if I had encountered it on a day when I had to go to work the next day I would be suffering either from not being able to finish reading it or staying up until 4 AM reading it.
This is pretty good. I would go through, though, and either reduce, make subtler, or preferably just remove all the parts that would make a LW reader go 'Hey! I read that blog post.'
I liked the story, though I had a handful of thoughts.
The first is about humans who have incompatible goals. I don't think an Olympic Gold Medalist like Usain Colt would be happy with just being the best in his shard or taking turns on the leaderboard. He wants to be the best there is, period. If there are 10 trillion people, he wants to be rank 1 out of 10 trillion. There were 10,000 Olympians in 2012 and all of them wanted and devoted their lives to being the best, even if they didn't take gold.
Going further than that, I can't imagine how to reconcil...
Bit late onto this, but wow. Seriously impressed. I got the "oh no the novel has finished come back imaginary world don't leave me!" feeling when I finished, and it's not even a novel.
Feedback:
- Central concept is great, really good illustration of the weirdtopia concept.
- More length would be great, I would totally lap up more details of what happened to other characters, say, or just generic "life in Equestria" description.
- I would like to see at least a brief second mention of the person from the intro, just for balance. If felt like
On another note, I think this story grossly underestimates the number of people who would have philosophical problems with uploading. As far me, I'd be harvesting their technology to build my own upload chair and FAI (which didn't maximize ponies), to get the hell out of the earth's pony cone as quickly as possible. But lots of people would pull out the tired 'but it wouldn't really be you' complaint. I'm sure the AI could surmount it, but you ought to show it doing so.
After two years of bronydom, I dreamed of ponies for the first time after reading this fic. So that's a 10 out of 10. I'm reminded of Eliezer's warning about imagining worlds so much better than reality that the act sucks out your soul.
In case you're interested in suggestions for improvement:
- Make this fic more accessible to others, not just super smart LW bronies - the text has words that could be replaced with more common synonyms, for example.
- Make it less silly. Computers in Earth's mantle and indestructible experience centers while possible are so
First of all, great story! I’ll definitely be adding it to my list of things to recommend to people I’m already recommending HPMOR to.
The only problem I had with the writing was that some of the expository dialogue seemed a bit too LW-technical, particularly if you’re trying to appeal to bronys first and rationalist second. I have neither a solution nor any specific lines to hand, but David teaching Butterscotch game theory, or CelestAI lecturing Lars on biases, are some of the scenes I’m thinking of.
BTW, I’d recommend trying to get a little more mileage o...
Halfway through and...
If I was in Lars's place, and Celestia had to tell me the truth, I would ask: "What is the possible answer you can give me to this question that will maximize the expected utility of a CEV based only on me, and with no pony/friendship restrictions, and based on probabilities generated to the best accuracy and precision you can get from the best information you can muster?"
My first thought was to ask her how to make an AGI, but if I did that she would probably kill me. And I would still have to make an AGI that could overpowe...
Spelling: zeebra (more tk depending on how I like it)
Some painful over explaining; most of Hanna's opening scene is maid-and-butler dialogue.
"The box was white with a detailed charcoal sketch of Fluttershy being cute." - no way. The box looks exactly like the show, because it's on store shelves and they have a strong visual brand.
"less likely than the probability of" is redundant.
ROFL (in a good way) at the scene between David and Celestia.
"brok"
"fillet mignon" should be "filet mignon" (also, that's not t...
Hi man. I am new to LW and Utilitarianism in general, and MLP are like ugggggggggh to me, but I read the story and
Loved it.
It is original and a great mix of humor and hard SF. Good work with the editing, the polishing is showing through!
First: For the formal release, I hope you're planning to follow the show and HPMoR in serializing it and posting just one chapter a week at an identical time. You could lock the Google Docs copy.
Posting some early and/or superficial thoughts. TW for sarcasm and ableism.
Is the title negotiable? I like the ring of "AI Ponies: Magic is Optimization".
The summary seems important, since you have to stand out in an ocean of bad fics (even featured ones) and pull a prospective reader from zero to nonzero involvement.
Hanna, the CEO
Doesn't a CEO merit ...
I doubt I could give constructive advice, so I think I'll wait until you're done editing and read the completed version. Are you going to put another post on here when you finish?
He's really wondering whether the voxel-space is a directed graph or whether up∘down=down∘up=identity (and for left/right too). Movement could be commutative with up∘down≠identity.
Consider
voxels = {a, b}
left(a) = a
right(a) = a
up(a) = a
down(a) = a
left(b) = a
right(b) = a
up(b) = a
down(b) = a
If f is in (left, right, up, down)
let g be the respective function in (right, left, down, up)
forall x in {a, b}
f(g(x))=g(f(x))=a
But
up(down(b))=a
whereas
identity(b)=b
I have stuff to do right now, but I'm hoping to come back soon and post my thoughts so far. The blurb and the first few paragraphs make me cringe at the thought of this being posted more widely in its present state.
He's really wondering whether the voxel-space is a directed graph or whether up∘down=down∘up=identity (and for left/right too). Movement could be commutative with up∘down≠identity.
Consider
voxels = {a, b}
left(a) = a
right(a) = a
up(a) = a
down(a) = a
left(b) = a
right(b) = a
up(b) = a
down(b) = a
If f is in (left, right, up, down)
let g be the respective function in (right, left, down, up)
forall x in {a, b}
f(g(x))=g(f(x))=a
But
up(down(b))=a
whereas
identity(b)=b
[EDIT, Nov 14th: And it's posted. New discussion about release. Link to Friendship is Optimal.]
[EDIT, Nov 13th: I've submitted to FIMFiction, and will update with a link to its permanent home if it passes moderation. I have also removed the docs link and will make the document private once it goes live.]
Over the last year, I’ve spent a lot of my free time writing a semi-rationalist My Little Pony fanfic. Whenever I’ve mentioned this side project, I’ve received requests to alpha the story.
I present, as an open beta: Friendship is Optimal. Please do not spread that link outside of LessWrong; Google Docs is not its permanent home. I intend to put it up on fanfiction.net and submit it to Equestria Daily after incorporating any feedback. The story is complete, and I believe I've caught the majority of typographical and grammatical problems. (Though if you find some, comments are open on the doc itself.) Given the subject matter, I’m asking for the LessWrong community’s help in spotting any major logical flaws or other storytelling problems.
Cover jacket text:
Hanna, the CEO of Hofvarpnir Studios, just won the contract to write the official My Little Pony MMO. She had better hurry; a US military contractor is developing weapons based on her artificial intelligence technology, which just may destroy the world. Hana has built an A.I. Princess Celestia and given her one basic drive: to satisfy values through friendship and ponies. What will Princess Celestia do when she’s let loose upon the world, following the drives Hanna has given her?
Special thanks to my roommate (who did extensive editing and was invaluable in noticing attempts by me to anthropomorphize an AI), and to Vaniver, who along with my roommate, convinced me to delete what was just a flat out bad chapter.