You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

KatjaGrace comments on Superintelligence 10: Instrumentally convergent goals - Less Wrong Discussion

7 Post author: KatjaGrace 18 November 2014 02:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (31)

You are viewing a single comment's thread.

Comment author: KatjaGrace 18 November 2014 02:01:22AM 3 points [-]

Can you think of more convergent instrumental values?

Comment author: SteveG 18 November 2014 06:41:46AM 1 point [-]

"Convergent Instrumental Values" seems like a very jargony phrase for "good tactics."

Patience Caution Deterred by credible threat Reputation-aware- It will extensively model what we think about it. Curiousity Reveals information when strategically appropriate ...

Comment author: TRIZ-Ingenieur 24 November 2014 09:33:24PM 0 points [-]

Your additional instrumental values spread your values and social influence become very important if we avoid rising of a decisive advantage AI.

In a society of AI enhanced humans and other AGIs altruism will become an important instrumental value for AGIs. The wide social network will recognize well-behavior and anti-social behavior without forgetting. Facebook gives a faint glimpse of what will be possible in future.

Stephen Omohundro said it in a nutshell: “AIs can control AIs.”