= 27d567bc2d48d9f40e9ccc20ea370b15)
waveman comments on Open thread, Oct. 03 - Oct. 09, 2016 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (168)
One perhaps useful analogy for super-intelligence going wrong is corporations.
We create corporations to serve our ends. They can do things we cannot do as individuals. But in subtle and not-so-subtle ways corporations can behave in very destructive ways. One example might be the way that they pursue profit at the cost of in some cases ruining people's lives, damaging the environment, corrupting the political process.
By analogy it seems plausible that super-intelligences may behave in a way that is against our interests.
It is not valid to assume that a super-intelligence will be smart enough to discern true human interests, or that it will be motivated to act on this knowledge.
Are you saying that no complex phenomenon is going to be able to provide only benefits and nothing but benefits, or are you saying that corporations are, on the balance, bad things and we would have been better to never have invented them?
No. Maybe it is possible. I am suggesting that it is not automatic that our creations serve our interests.
No. Saying something has harmful effects is not the same as saying that it is overall bad.
I am illustrating ways in which our creations can fail to serve our interests.
They do not have to be onmiscient to be smarter in some respects than human individuals.
It is hard to control their actions and to make sure they do serve our interests.
These effects can be subtle and difficult to understand.
But are corporations existiential threats?