niceguyanon comments on Open Thread, Aug. 22 - 28, 2016 - Less Wrong

3 Post author: polymathwannabe 22 August 2016 04:24PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (67)

You are viewing a single comment's thread. Show more comments above.

Comment author: Lumifer 24 August 2016 03:43:15PM 1 point [-]

AI gets it wrong then well that's it.

Not necessarily, depends on your AI and how god-like it is.

In the XIX century you could probably make the same argument about corporations: once one corporation rises above the rest, it will use its power to squash competition and install itself as the undisputed economic ruler forever and ever. The reality turned out to be rather different and not for the lack of trying.

Comment author: niceguyanon 24 August 2016 06:37:22PM 0 points [-]

Not necessarily, depends on your AI and how god-like it is.

I hope you're right. I just automatically think that AI is going to be god-like by default.

In the XIX century you could probably make the same argument about corporations

Not just corporations; you could make the some argument for sovereign states, foundations, trusts, militaries, and religious orgs.

Weak argument is that corporations with their visions, charters, and mission statements are ultimately run by a meatbag or run jointly by meatbags that die/retire, at least that's how it currently is. You can't retain humans forever. Corporations lose valuable and capable employee brains over time and replace them with new brains, which maybe better or worse, but you certainly cant keep your best humans forever. Power is checked; Bill Gates plans his legacy, while Sumner Redstone is infirm with kids jockeying for power and Steve Jobs is dead.

Comment author: Lumifer 24 August 2016 06:47:20PM 1 point [-]

AI is going to be god-like by default

Well, the default on LW is EY's FOOM scenario where an AI exponentially bootstraps itself into Transcendent Realms and, as you say, that's it. The default in the rest of the world... isn't like that.