= 27d567bc2d48d9f40e9ccc20ea370b15)
solipsist comments on Expected utility, unlosing agents, and Pascal's mugging - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (54)
And the utility function doesn't have to be bounded by a constant. An agent will "blow out its speakers" if it follows a utility function whose dynamic range is greater than the agent's Kolmogorov complexity + the evidence the agent has accumulated in its lifetime. The agent's brain's subjective probabilities will not have sufficient fidelity for such a dynamic utility function to be meaningful.
Super-exponential utility values are OK, if you've accumulated a super-polynomial amount of evidence.
The Solomonoff expectation value of any unbounded computable utility function diverges. This is because the program "produce the first universe with utility > n^2" is roughly of length log n + O(1) therefore it contributes 2^{-log n + O(1)} n^2 = O(n) to the expectation value.
Oops, that's not quite right. But I think that something like that is right :-). 15 Quirrell points to whoever formalizes it correctly first.