= 27d567bc2d48d9f40e9ccc20ea370b15)
A113 comments on Think Twice: A Response to Kevin Kelly on ‘Thinkism’ - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (12)
I agree with both you and Kelly most of the time, you more than him. I did think this part required a nitpick:
If I became superintelligent tomorrow, I probably wouldn't significantly change the world. Not on a Singularity scale, not right away, and not just because I could. Would you? My point there is that you can't assume that because the first superintelligence can construct nanobots and take over the world, it therefore will.
A lot depends on what we mean by "superintelligent." But yes, there's a level of intelligence above which I'm fairly confident that I would change the world, as rapidly as practical, because I can. Why wouldn't you?
Not just because I can. Maybe for other reasons, like the fact that I still care about the punier humans and want to make it better for them. That depends on preferences that an AI might or might not have.
It's not really about what I would do; it's the fact that we don't know what an arbitrary superintelligence will or won't decide to do.
(I'm thinking of "superintelligence" as "smart enough to do more or less whatever it wants by sheer thinkism," which I've already said I agree is possible. Is this nonstandard?)
Sure, "because I have preferences which changing the world would more effectively maximize than leaving it as it is" is more accurate than "because I can". And, sure, maybe an arbitrary superintelligence would have no such preferences, but I'm not confident of that.
(Nope, it's standard (locally).)