You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

EE43026F comments on The Stable State is Broken - Less Wrong Discussion

57 Post author: Bakkot 12 March 2012 06:31PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (43)

You are viewing a single comment's thread.

Comment author: EE43026F 12 March 2012 07:21:37PM 11 points [-]

That's a very important and basic observation.

You missed another example : cancer. Cankerous cells are much better at replicating themselves than normal cells are. Pluricellular organisms have a multitude of systems to keep their component cells in check, yet they still fail at it from time to time. Biology has had billions of years of evolution to fine tune how it enforces cooperation within larger organisms. Can we do better, especially as the components we're considering at our scale may be as complex and clever to us as cells are to an organism? (Meaning we may not have a comparative advantage even though we're subtler than evolution).

So aside from asking what we can do next, I'd like to add : "Can we do something next?" In order to enforce a system within which you won't observe such an effect, you might need to be larger, have more resources than the sum of all you're trying to steer. Otherwise, some part of that system will eventually take over. And even then, chance events may always remain beyond your capacity to control.

Comment author: Mass_Driver 13 March 2012 06:38:45AM 1 point [-]

Cancer (and anti-cancer immune systems) might be a very fruitful analogy. To fight the tendency of systems to fall toward a stable state of suboptimal selfishness and shallowness, it might help to explicitly punish self-promotion or explicitly reward competence.

Something like the former happened in America during the Progressive Era of the 1900s and 1910s, when racketeers and robber barons were thrown out of the offices they'd schemed their way into by a cadre of self-appointed elitist technocrats.

Something like the latter happened in the 1940s and 1950s, when IQ tests, the SAT, vast increases in education expenditure, cracks in the wall of WASP solidarity, and major construction and infrastructure programs put meritocratic engineers at the top of many corporations and agencies.

Comment author: Eugine_Nier 13 March 2012 11:14:32PM 2 points [-]

it might help to explicitly punish self-promotion or explicitly reward competence.

The problem is that then you have to keep the "reward and punish" system itself from corrupting.