You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

orthonormal comments on The Stable State is Broken - Less Wrong Discussion

57 Post author: Bakkot 12 March 2012 06:31PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (43)

You are viewing a single comment's thread. Show more comments above.

Comment author: fiddlemath 13 March 2012 03:48:21PM 12 points [-]

I agree, this class of problems is enormous, and has a hand in practically all mismanagement and misdirection of human effort. The problem is so aggravating, though, because humans seem not to expect it to happen. Why do these feel like "problems", when the underlying behavior is exactly what we'd expect given our knowledge of stable strategy?

In the case of human endeavor, I suspect this is a problem because we do not try hard enough to defend against it. We seem surprised and indignant when systems that purport to do good are "gamed." If more people were more aware that this was the consequence of strategic agents, they might watch harder for signs of destructive strategy, and more carefully design the systems that they build and manage against such strategy.

(Hmm. I notice that I'm posing a purportedly-fully-general strategy to a fully-general problem, without evidence or examples. And I'm claiming that better global understanding is a solution, when it is probably just applause lights to my imagined audience. And I still think that what I'm saying is right! Wow. You should probably ignore this comment's content, but I'll still leave it here, as a counterexample.)

Comment author: orthonormal 13 March 2012 10:16:08PM 5 points [-]

Also because our ancestral environment didn't have large enough selection pools for the social and economic examples to arise- we could blame any failures in the tribe on particular people, and so we think of such things as vice rather than obvious selection bias.