= 27d567bc2d48d9f40e9ccc20ea370b15)
patrissimo comments on The danger of living a story - Singularity Tropes - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (61)
So far as SIAI is concerned, I have to say that the storylike qualities of the situation have provided no bonus at all to our PR rolls, just a penalty we have to be careful to avoid because of all the people who perceptually recognize it as a mere story. In other words, we lose because of all the people trying to compensate for a bias that doesn't actually seem to be there. People who really are persuaded by stories, go off and become religious or something, find people with much better-refined attractive stories than us. Our own target audience, drawn from the remainder, tends to see any assertion classifiable as storylike as forbidden-in-reality because so many stupid people believe in them.
And of course much of your story simply and obviously isn't applicable. There will be no robot war, there don't seem to be any hostile human parties trying to bring about the apocalypse either, the question gets resolved in a research basement somewhere and then there's no final battle one way or another.
But then if you'd left out the part about the Robot War and the final battle, your opening paragraph wouldn't have sounded as clever. And this is also something that happens to us a LOT, which is that people arguing against us insist on mapping the situation very exactly onto a story outline, at the expense of accuracy, so that it looks stupider.
All the bias here seems to be in the overcompensation.
Actually, I will comment (for the purposes of authenticity and from the belief that being more transparent about my motivations will increase mutual truth-finding) that while I'm not arguing "against" SIAI, this post is to some degree emerging from me exploring the question of SIAI's organizational instrumental rationality. I have the impression from a variety of angles/sources that it's pretty bad. Since I care about SIAI's success, it's one of the things I think about in the background - why, and how you could be more effective.
When discussing SIAI's instrumental rationality, it's important to remember what its actual goals are. Speaking of story-bias, it's all too easy to pattern-match to "organization promoting some cause they think is important", in which case one easily concludes that SIAI has been a miserable failure because FAI hasn't become a trendy academic research discipline, and Vice Presidents aren't making films about paperclip maximizers.
However, the picture changes somewhat if instead you think in terms of the following (more accurate) caricature of SIAI's actual objectives:
(1) To persuade a dozen or so Putnam Fellows to collaborate with Eliezer on FAI instead of pursuing brilliant careers in academic mathematics;
(2) To dissuade people like Ben Goertzel from trying to build AGI without solving the FAI problem first.
If you look at it like this (still admittedly oversimplified), then yes, SIAI still has a way to go in achieving its goals, but they don't seem to be quite as hopelessly underequipped for the task as one might have thought.
(Disclaimer: I certainly don't speak for SIAI; my association with the organization is that of a former visitor, i.e. about as loose as it's possible to get while still having to answer "yes" to the question "Are you, or have you ever been, affiliated with the Singularity Institute for Artificial Intelligence?" if ever called to testify before Congress....)
I've had similar thoughts. I would be interested in hearing what room for improvement you see in SIAI's organizational instrumental rationality. I have my own thoughts on this (which have evolved somewhat as I've learned more since making my posts about SIAI back in August). Feel free to PM me if you'd prefer to communicate privately.
You'll want to see my reply to patrissimo above.