timtyler comments on Amending the "General Pupose Intelligence: Arguing the Orthogonality Thesis" - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (22)
Sure, but there are cases where rivals are evenly matched. Lions and tigers, for instance, have different - often conflicting - aims. However, it isn't a walk-over for one team. Of course, you could say whether the lion or tiger genes win is "just a convention" - but to the lions and tigers, it really matters.
No use. However, our values are not that far from Universal Instrumental Values - because we were built by a process involving a lot of natural selection.
Our choice is more like: do we give up a few of the things we value now - or run the risk of losing many more of them in the future. That leads to the question of how big the risk is - and that turns out to be a tricky issue.
Agreed. That tricky issue I suspect might have enormous consequences if reason ends up being highjacked by in-group out-group biases, and the surviving memes end up being those that make us more instrumental, for fear of someone else doing the same.
I expect that the force that will eventually promote natural values most strongly will be the prospect of encountering unknown aliens. As you say, the stakes are high. If we choose incorrectly, much of our distinctiveness could be permanently obliterated.