You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

shminux comments on Stupid questions thread, October 2015 - Less Wrong Discussion

3 Post author: philh 13 October 2015 07:39PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (223)

You are viewing a single comment's thread. Show more comments above.

Comment author: shminux 14 October 2015 04:19:25PM 0 points [-]

An AI designed to be friendly/aligned probably would not. Having a layer not available for extensive introspection is a big risk. On the other hand, if you just let a neural net run wild, then all bets are off.

Comment author: NancyLebovitz 14 October 2015 06:07:22PM 1 point [-]

It depends on how you define system 1. An AI might have a system 1 for fast reactions, but also have more ability to observe and change their system 1 than humans do.