patrissimo comments on The danger of living a story - Singularity Tropes - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (61)
So far as SIAI is concerned, I have to say that the storylike qualities of the situation have provided no bonus at all to our PR rolls, just a penalty we have to be careful to avoid because of all the people who perceptually recognize it as a mere story. In other words, we lose because of all the people trying to compensate for a bias that doesn't actually seem to be there. People who really are persuaded by stories, go off and become religious or something, find people with much better-refined attractive stories than us. Our own target audience, drawn from the remainder, tends to see any assertion classifiable as storylike as forbidden-in-reality because so many stupid people believe in them.
And of course much of your story simply and obviously isn't applicable. There will be no robot war, there don't seem to be any hostile human parties trying to bring about the apocalypse either, the question gets resolved in a research basement somewhere and then there's no final battle one way or another.
But then if you'd left out the part about the Robot War and the final battle, your opening paragraph wouldn't have sounded as clever. And this is also something that happens to us a LOT, which is that people arguing against us insist on mapping the situation very exactly onto a story outline, at the expense of accuracy, so that it looks stupider.
All the bias here seems to be in the overcompensation.
I can see how for your audience, the story-like qualities would be a minus. On the other hand, I think the story bias has to do with how people cognitively process information and arguments. If you can't tell your mission & strategy as a story, it's a lot harder to get across your ideas, whatever your audience.
The battle was meant to be metaphorical - the battle to ensure that AI is Friendly rather than Unfriendly. And I didn't say anything about hostile humans - the problem is indifferent humans not giving you resources.
Also, I'm not arguing against SIAI, I just find it amusing how well the futurist sector maps onto a story outline - various protagonists passionate about fighting some great evil that others don't see and trying to build alliances and grow resources before time runs out. You can squiggle, but that's who you are. Instrumental rationality means figuring out how to make best positive use of it and avoid it biasing you.
By this standard, just about anything worth taking as one's life work will involve a metaphorical battle.