Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Stuart_Armstrong comments on The Importance of Goodhart's Law - Less Wrong

75 Post author: blogospheroid 13 March 2010 08:19AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (113)

You are viewing a single comment's thread.

Comment author: Stuart_Armstrong 14 March 2010 12:41:55PM 16 points [-]

CEV, until designed and defined properly, is just a black box that everyone universally agrees is 'good', but has little else in term of defining features.

Comment author: jimmy 14 March 2010 07:08:33PM 2 points [-]

It's not so much a box, but a method of filling the box. We just haven't filled the box yet.

Comment deleted 29 March 2010 08:43:37PM [-]
Comment author: Mass_Driver 05 April 2010 02:01:30PM 2 points [-]

If the question is "what should we want?" then CEV is much better than a black box, because it fleshes out some of the intuitions behind the magical category "want."

If the question is "how should we measure what we want?" then CEV is just a black box, because it doesn't solve or suggest a method for solving any of our measurement problems. We know we want coherent, extrapolated, volitional thingies, but we have no idea, for example, how to rigorously define "volitional." We likewise have no idea how far into the future we should be extrapolating things, nor how many facets of a personality or society can reasonably be expected to converge/cohere.