pengvado comments on Causal decision theory is unsatisfactory - Less Wrong

20 Post author: So8res 13 September 2014 05:05PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (158)

Comments (158)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 16 September 2014 05:30:12PM *  1 point [-]

You're correct that this includes problems where UDT performs poorly, and that UDT is by no means the One Final Answer.

What problems does UDT fail on?

my goal is to motivate the idea that we don't know enough about decision theory yet to be comfortable constructing a system capable of undergoing an intelligence explosion.

Why would a self-improving agent not improve its own decision-theory to reach an optimum without human intervention, given a "comfortable" utility function in the first place?

Comment author: pengvado 18 September 2014 08:42:44AM 3 points [-]

Why would a self-improving agent not improve its own decision-theory to reach an optimum without human intervention, given a "comfortable" utility function in the first place?

A self-improving agent does improve its own decision theory, but it uses its current decision theory to predict which self-modifications would be improvements, and broken decision theories can be wrong about that. Not all starting points converge to the same answer.

Comment author: [deleted] 22 September 2014 05:20:43PM 1 point [-]

Oh. Oh dear. DERP. Of course: the decision theory of sound self-improvement is a special case of the decision theory for dealing with other agents.

Powered by Reddit Powered by Reddit