= 27d567bc2d48d9f40e9ccc20ea370b15)
Wei_Dai comments on The AI design space near the FAI [draft] - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (48)
I think this is a serious concern, especially as I'm starting to suspect that AGI might require decision theoretic insights about reflection in order to be truly dangerous. If my suspicion is wrong then SingInst working directly on FAI isn't that harmful marginally speaking, but if it's right then SingInst's support of decision theory research might make it one of the most dangerous institutions around.
Given that you're worried and that you're highly respected in the community, this would seem to be one of those "stop, melt, and catch fire" situations that Eliezer talks about, so I'm confused about SingInst's apparently somewhat cavalier attitude. They seem to be intent on laying the groundwork for the ennead.
Another way in which decision theoretic insights may be harmful is if they increase the sophistication of UFAI and allow them to control less sophisticated AGIs in other universes.
I'm trying to avoid being too confrontational, which might backfire, or I might be wrong myself. It seems safer to just push them to be more strategic and either see the danger themselves or explain why it's a good idea despite the dangers.