Voted up as this is an important and general question about converting human intuitions into a formal utility function.
What makes the desire to obtain high status within some small group a legitimate piece of Godshatter (good), as opposed to a kind of scope insensitivity (bad)?
Do we have a general criterion for deciding these things? Or is it still unresolved in general?
In this specific case, it seems to me that there are many aspects of social interaction that are zero-sum or even negative sum. For the purpose of Coherent Extrapolated Volition, zero sum or negative sum elements are like scope insensitivity, i.e. bad.
There are clearly some social status games that are positive sum.
Do we have a general criterion for deciding these things? Or is it still unresolved in general?
I think it's unresolved in general. I brought up scope insensitivity as a counter-example to the "Godshatter" argument, or at least a strong form of it which says we should keep all of the values that evolution has handed down to us. It seems likely that we shouldn't, but exactly where to draw the line is unclear to me. Still, to me, desire for high status in some small group seems to be the same kind of "crazy" value as scope insensitivity...
Followup to: Do you have High-Functioning Asperger's Syndrome?
LW reader Madbadger uses the metaphor of a GPU and a CPU in a desktop system to think about people with Asperger's Syndrome: general intelligence is like a CPU, being universal but only mediocre at any particular task, whereas the "social coprocessor" brainware in a Neurotypical brain is like a GPU: highly specialized but great at what it does. Neurotypical people are like computers with measly Pentium IV processors, but expensive Radeon HD 4890 GPUs. A High-functioning AS person is an Intel Core i7 Extreme Edition - with on-board graphics!
This analogy also covers the spectrum view of social/empathic abilities, you can think about having a weaker social coprocessor than average if you have some of the tendencies of AS but not others. You can even think of your score on the AQ Test as being like the Tom's Hardware Rating of your Coprocessor. (Lower numbers are better!).
If you lack that powerful social coprocessor, what can you do? Well, you'll have to run your social interactions "in software", i.e. explicitly reason through the complex human social game that most people play without ever really understanding. There are several tricks that a High-functioning AS person can use in this situation: