steven0461 comments on How can we ensure that a Friendly AI team will be sane enough? - Less Wrong

10 Post author: Wei_Dai 16 May 2012 09:24PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (64)

You are viewing a single comment's thread. Show more comments above.

Comment author: steven0461 17 May 2012 02:15:43AM *  1 point [-]

I meant to refer to just bug fixes, I think. My comment wasn't really responsive to yours, just prompted by it, and I should probably have added a note to that effect. One can imagine a set of bugs that become more fixed or less fixed over time, varying together in a continuous manner, depending on e.g. what emotional state one is in. One might be more vulnerable to many bugs when sleepy, for example. One can then talk about averages and extreme values of such a "general rationality" factor in a typical decision context, and talk about whether there are important non-standard contexts where new bugs become important that one hasn't prepared for. I agree that bugs related to status (and to interpersonal conflict) seem particularly dangerous.