timtyler comments on Model Uncertainty, Pascalian Reasoning and Utilitarianism - Less Wrong

23 Post author: multifoliaterose 14 June 2011 03:19AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (154)

Comments (154)

You are viewing a single comment's thread. Show more comments above.

Comment author: TimFreeman 16 June 2011 02:35:23PM 3 points [-]

Why are people on Less Wrong still talking about 'their' 'values' using deviations from a model that assumes they have a 'utility function'? It's not enough to explicitly believe and disclaim that this is obviously an incorrect model, at some point you have to actually stop using the model and adopt something else. People are godshatter, they are incoherent, they are inconsistent, they are an abstraction, they are confused about morality, their revealed preferences aren't their preferences, their revealed preferences aren't even their revealed preferences, their verbally expressed preferences aren't even preferences, the beliefs of parts of them about the preferences of other parts of them aren't their preferences, the beliefs of parts of them aren't even beliefs, preferences aren't morality, predisposition isn't justification, et cetera...

We might make something someday that isn't godshatter, and we need to practice.

I agree that reforming humans to be rational is hopeless, but it is nevertheless useful to imagine how a rational being would deal with things.

Comment author: timtyler 16 June 2011 09:59:20PM -1 points [-]

Since even irrational agents can be modelled using a utility function, no "reforming" is needed.

Comment author: jsteinhardt 16 June 2011 10:43:25PM 1 point [-]

How can they be modeled with a utility function?

Comment author: timtyler 17 June 2011 07:08:43AM 2 points [-]

As explained here:

Any agent can be expressed as an O-maximizer (as we show in Section 3.1)

Comment author: jsteinhardt 17 June 2011 08:59:02PM 1 point [-]

Thanks for the reference.

It seems though that the reward function might be extremely complicated in general (in fact I suspect that this paper can be used to show that the reward function can be potentially uncomputable).

Comment author: timtyler 17 June 2011 09:50:16PM 0 points [-]

The whole universe may well be computable - according to the Church–Turing–Deutsch principle. If it isn't the above analysis may not apply.

Comment author: TimFreeman 17 June 2011 10:33:28PM 0 points [-]

I agree with jsteinhardt, thanks for the reference.

I agree that the reward functions will vary in complexity. If you do the usual thing in Solomonoff induction, where the plausibility of a reward function decreases exponentially with its size, so far as I can tell you can infer reward fuctions from behavior, if you can infer behavior.

We need to infer a utility function for somebody if we're going to help them get what they want, since a utility function is the only reasonable description I know of what an agent wants.

Powered by Reddit Powered by Reddit