Douglas_Reay comments on The Ape Constraint discussion meeting. - Less Wrong

9 Post author: Douglas_Reay 28 November 2013 11:22AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (23)

You are viewing a single comment's thread. Show more comments above.

Comment author: Douglas_Reay 28 November 2013 02:38:16PM -2 points [-]

wasn't it agreed-upon by FAI researchers that "Friendliness" as a supergoal meant that the AI would find ways to do things that are "better" for humanity overall in its prediction of the grand schemes of things.

This would include "being a good galactic citizen" with no specific preference for humanity if the freedom, creativity, fairness, public perception by aliens, or whatever other factor of influence led this goal to being superior in terms of achieving human values and maximizing collective human utility.

I'm glad to hear it.

But I think there is a distinction here worth noting, between two positions:

POSITION ONE - Make "Be a good galactic citizen" be the supergoal if and only if setting that as the supergoal is the action that maximises the chances of the AI, in practice, ending up doing stuff to help humanity in the long term, once you take interfering aliens, etc into account

and

POSITION TWO - Make "Be a good galactic citizen" be the supergoal, even if that isn't quite as certain an approach to helping humanity in particular, as setting "be friendly to humanity" as the supergoal would be.

Why on earth would anyone suggest that AI researchers follow an approach that isn't the absolute safest for humanity? That's a big question. But one I think worth considering, if we open the possibility that there is a bit of wiggle room for setting a supergoal that will still be ok for humanity, but be slightly more moral.

Comment author: Gunnar_Zarncke 28 November 2013 02:46:19PM *  0 points [-]

You know: You don't need to comment your post if you want to extend it. You may edit it. It seems customary to add it with a tag like

EDIT: ...

--

Sorry. You are more senior than me. I confused you with a newbie. You will have your reason