Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Psy-Kosh comments on Free to Optimize - Less Wrong

25 Post author: Eliezer_Yudkowsky 02 January 2009 01:41AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (80)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Psy-Kosh 02 January 2009 04:53:48AM 0 points [-]

Yes, that's kind of my point: a "meddling god" of the classic "engaged in behavior that at least looked like it arose from human motivations" is something that a human can at least reasonably easily understand.

But rules arising from an alien "mind", rules that aren't simple either on a fundamental level or simple in a "simple relative to us" sense is something very different, not looking to us at all like a human judge making decisions.

Or am I completely and utterly missing the point here? (Don't misunderstand. I'm not saying that it is absolutely undesirable for things, at least initially, to work out as you suggest. But it does seem to me that there'd be a bit of an "understandability" cost, at least initially.)