John_Maxwell_IV comments on We Don't Have a Utility Function - Less Wrong

43 [deleted] 02 April 2013 03:49AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (123)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 02 April 2013 05:08:18AM 7 points [-]

Read lots, think lots, do lots.

More specifically, become convinced of consequentialism so that pragmatic concerns and exceptions can be handled in a principled way, realize that rule by Friendly AI would be acceptable, attempt to actually run a LW meetup and learn of the pragmatic effectiveness of central decision making, notice major inconsistencies in and highly suspect origin of my non-authoritarian beliefs, notice aesthetic value of leadership and following, etc.

Comment author: John_Maxwell_IV 02 April 2013 09:37:47AM *  7 points [-]

"Authority" isn't necessarily just one thing. For example, an all-powerful Friendly AI could choose to present itself in an extremely deferential way, and even conform exactly to it's human users' wishes. Being a central decisionmaker, projecting high status, having impressive accomplishments, having others feel instinctively deferential to you, and having others actually act deferential to you are all distinct but frequently related. I think at least some of these are worrisome (link).

If you increase the authority of a group's leader along all the dimensions of authority (which probably happens by default), I'd guess you get increased group coherence at the expense of decreased group rationality. You also run the risk of having the leader's preferences be satisfied at the expense of the group's preferences. In situations where it doesn't actually matter what you do much and it mostly just matters that everyone does it together in an orderly way, maybe this can be a good trade-off.