KatjaGrace comments on Superintelligence 10: Instrumentally convergent goals - Less Wrong

7 Post author: KatjaGrace 18 November 2014 02:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (31)

You are viewing a single comment's thread.

Comment author: KatjaGrace 18 November 2014 02:01:22AM 3 points [-]

Can you think of more convergent instrumental values?

Comment author: SteveG 18 November 2014 06:41:46AM 1 point [-]

"Convergent Instrumental Values" seems like a very jargony phrase for "good tactics."

Patience Caution Deterred by credible threat Reputation-aware- It will extensively model what we think about it. Curiousity Reveals information when strategically appropriate ...

Comment author: TRIZ-Ingenieur 24 November 2014 09:33:24PM 0 points [-]

Your additional instrumental values spread your values and social influence become very important if we avoid rising of a decisive advantage AI.

In a society of AI enhanced humans and other AGIs altruism will become an important instrumental value for AGIs. The wide social network will recognize well-behavior and anti-social behavior without forgetting. Facebook gives a faint glimpse of what will be possible in future.

Stephen Omohundro said it in a nutshell: “AIs can control AIs.”