= 27d567bc2d48d9f40e9ccc20ea370b15)
nyan_sandwich comments on We Don't Have a Utility Function - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (123)
Yes. "(Real bounded decision systems will take shortcuts for efficiency and may not achieve perfect rationality, like how real floating point arithmetic isn't associative)."
On one hand, a lot of this is lacking a proper theory of logical uncertainty, which a lot of this is (I think).
On the other hand, the usual solution is to step up a level to choose best decision algorithm instead of trying to directly compute best decision. Then you can step up to not taking forever at this. I don't know how to bottom this out.
Related: A properly built AI need not do any explicit utility maximizing at all; it could all be built implicitly into hardcoded algorithms, the same way most algorithms have implicit probability distributions. Of course, one of the easiest ways to maximize expected utility is to explicitly do so, but I would still expect most code in an optimized AI to be implicitly maximizing.
What you need to estimate for maximizing the utility is not utility but sign of the difference in expected utilities. "More accurate" estimation of utility on one side of the comparison can lead to less accurate estimation of the sign of the difference. Which is what Pascal muggers use.
This is a very good point.
I wonder what the implications are...
The main implication is that actions based on comparison between most complete available estimations of utility do not maximize utility. It is similar to evaluating sums; when evaluating 1-1/2+1/3-1/4 and so on, the 1+1/3+1/5+1/7 is a more complete sum than 1 - you have processed more terms (and can pat yourself on the head for doing more arithmetics) , but less accurate. In practice one obtains highly biased "estimates" from someone putting a lot more effort into finding terms of the sign that benefits them the most, and sometimes, from some terms being easier to find.
Yes, that is a problem.
Are there other schemes that do a better job, though?
In the above example, attempts to produce a most accurate estimate of the sum do a better job than attempts to produce most complete sum.
In general what you learn from applied mathematics is that plenty of methods that are in some abstract sense more distant from the perfect method have a result closer to the result of the perfect method.
E.g. the perfect method could evaluate every possible argument, sum all of them, and then decide. The approximate method can evaluate a least biased sample of the arguments, sum them, and then decide, whereas the method that tries to match the perfect method the most would sum all available arguments. If you could convince an agent that the latter is 'most rational' (which may be intuitively appealing because it does resemble the perfect method the most) and is what should be done, then in a complex subject where agent does not itself enumerate all arguments, you can feed arguments to that agent, biasing the sum, and extract profit of some kind.
"Taken together the four experiments provide support for the Sampling Hypothesis, and the idea that there may be a rational explanation for the variability of children’s responses in domains like causal inference."
That seems to be behind what I suspect is a paywall, except that the link I'd expect to solicit me for money is broken. Got a version that isn't?
It's going through a university proxy, so it's just broken for you. Here's the paper: http://dl.dropboxusercontent.com/u/85192141/2013-denison.pdf