Okay, I'm with you so far. But what I was actually asking for was an example of a scenario where this wrapping gives us some benefit that we wouldn't have otherwise.

I don't think utility functions are a very good tool to use when seeking to clarify one's goals to yourself. Things like PJ Eby's writings have given me rather powerful insights to my goals, content which would be pointless to try to convert to the utility function framework.

Alright, I fortunately haven't worked with Dr. Evils, so I'll defer to your experience.

As for how Dr. Evil might know he was under a threat of torture, it was stated in the paper that he received a message from the Philosophy Defence Force telling him he was. It was also established that the Philosophy Defence Force never lies or gives misleading information. ;)

(I, myself, haven't received any threats from organizations known to never lie or be misleading.)

I don't think I understand what you're trying to describe here. Could you give an example of a scenario where you usefully transform a model into a utility-based one the way you describe?

I'm not bashing utility-based models, I'm quite aware of their good sides. I'm just saying they shouldn't be used universally and without criticism. That's not bashing any more than it's bashing to say that integrals aren't the most natural way to do matrix multiplication with.

I certainly wouldn't want to punish them, which is why I'd also upvote any such thread that ended up with a negative score.

ETA: It would seem that rationality quotes are no longer desired. After several days this thread stands voted into the negatives. Wolud whoever chose to to downvote this below 0 would care to express their disapproval of the regular quotes tradition more explicitly?

For the record, I didn't downvote this below zero, but it did at one point downvote this back to zero (and did the same for the Open Thread). Not because I'd disagree with the tradition in any way, but because I don't think the first person to get around posting the month's thread should get tens of points of karma for simply being quick.

Huh. Reading this comment again, I realize I've shifted considerably closer to your view, while forgetting that we ever had this discussion in the first place.

I've been thinking a lot lately of treating ethical theories as models...

Any relation to my thoughts of ethical theories as models?

http://lesswrong.com/lw/18l/ethics_as_a_black_box_function/

http://lesswrong.com/lw/18l/ethics_as_a_black_box_function/14ha

Well, yes, "so complex and chaotic that you might as well call it random" is what I meant. That's what's usually meant by the term - the results of dice rolls aren't mainly driven by quantum randomness either.

Wait, do you mean that your construction is simply acting as a wrapper on some underlying model, and converting the outputs of that model into a different format?

If that's what you mean, then well, sure. You could do that without noticeably increasing the complexity. But in that case the utility wrapping doesn't really give us any useful additional information, and it'd still be the underlying model we'd be mainly interested in.

Yes, I admit that it can sometimes be useful to think of humans as having utility functions, and this can be a useful model. I should have said that in the post, now that you mention it. But then one should then always keep in mind that that's just a simplified model that's appropriate for certain situations, not something that can be indiscriminately employed in every case.