Wei_Dai comments on Ingredients of Timeless Decision Theory - Less Wrong

43 Post author: Eliezer_Yudkowsky 19 August 2009 01:10AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (226)

You are viewing a single comment's thread. Show more comments above.

Comment author: Wei_Dai 20 August 2009 07:29:52PM *  0 points [-]

refuse to formalize

What about what I wrote?

These considerations lead to the following design for the decision algorithm S. S is coded with a vector <P1, P2, P3, ...> of programs that it cares about, and a utility function on vectors of the form <E1, E2, E3, …> that defines its preferences on how those programs should run. When it receives an input X, it looks inside the programs P1, P2, P3, ..., and uses its "mathematical intuition" to form a probability distribution P_Y over the set of vectors <E1, E2, E3, …> for each choice of output string Y. Finally, it outputs a string Y* that maximizes the expected utility Sum P_Y(<E1, E2, E3, …>) U(<E1, E2, E3, …>).

Which part do you find insufficiently formal? Of course I use "mathematical intuition" as a black box without explaining how it works, but that's just like EDT using "prior" without explaining where it comes from, or CDT using "causal probability" as a black box. It's an unsolved problem, not refusal to formalize.

Comment author: cousin_it 20 August 2009 07:34:53PM *  0 points [-]

Your decision theory is formal enough for me, but it seems to be different from Eliezer's, which I was talking about. If they're really the same, could you explain how?

Comment author: Wei_Dai 20 August 2009 08:07:58PM 0 points [-]

In that case, I never said I understood Eliezer's version well enough that I could formalize it if I wanted to, and I don't think Nesov and Drescher claimed that either, so I don't know why you mentioned our names in connection with "refuse to formalize". Actually I explicitly said that I don't understand Eliezer's theory very well yet.

Comment author: cousin_it 20 August 2009 08:25:50PM *  0 points [-]

You're right. I apologize. Amended the comment.