= 27d567bc2d48d9f40e9ccc20ea370b15)
Essentially every post would have been better if it had included some additional thing. Based on various recent comments I was under the impression that people want more posts in Discussion so I've been experimenting with that, and I'm keeping the burden of quality deliberately low so that I'll post at all.
Let me rephrase: would you like to describe your arguments against utility functions in more detail?
For example, as I mentioned, there's an obvious mathematical equivalence between making a plan at the beginning of time and planning as you go, which is directly analogous to how one converts games from extensive-form to normal-form. As such, all aspects of acquiring information is handled just fine (from a mathematical standpoint) in the setup of vNM.
The standard response to the discussion of knowing probabilities exactly and to concerns about computational complexity (in essence) is that we may want to throw aside epistemic concerns and simply learn what we can from a theory that is not troubled by them (a la air resistance in physics..)? Is your objection essentially that those factors are more dominant in human morality than LW acknowledges? And if so, is the objection to the normal-form assumption essentially the same?
a utility function is the structure any consistent preference ordering that respects probability must have.
This is the sort of thing I mean when I say that people take utility functions too seriously. I think the von Neumann-Morgenstern theorem is much weaker than it initially appears. It's full of hidden assumptions that are constantly violated in practice, e.g. that an agent can know probabilities to arbitrary precision, can know utilities to arbitrary precision, can compute utilities in time to make decisions, makes a single plan at the beginning of time about how they'll behave for eternity (or else you need to take into account factors like how the agent should behave in order to acquire more information in the future and that just isn't modeled by the setup of vNM at all), etc.
I think your original post would have been better if it included any arguments against utility functions, such as those you mention under "e.g." here.
Besides being a more meaningful post, we would also be able to discuss your comments. For example, without more detail, I can't tell whether your last comment is addressed sufficiently by the standard equivalence of normal-form and extensive-form games.
(without the swap rule)
I'd like to point out that without the swap rule it's also very easy to write a program that plays perfectly.
I don't believe your comment is true in any meaningful sense. Can you explain what you mean?
Details: It's easy to prove that the first player wins in Hex without the swap rule, but it's even easier to prove the second wins in any (deterministic, ...) game with the swap rule. Neither proof is constructive, and so neither provides an efficient program.
Interpreting your statement differently, it's easy to write a program that plays any (deterministic, ...) game optimally. Just explore the full game tree! The program won't terminate for a while, however, and this interpretation makes no distinction between the versions with and without the swap rule.
I was a bit irritated that no fewer than three players submitted CooperateBot, as I cooperated with it in hopes of tricking bots like submission B.
EDIT: More offense was taken at my use of the word "trolls" than was intended.
Why does submitting CooperateBot to a competition that does not include it make someone a troll? Would submitting DefectBot make one a troll, too?
(I believe the competition should have automatically included one CooperateBot and DefectBot each, and stated that this was the case at the beginning. I am sad there were three CooperateBots and no DefectBots.)
In response to
What bothers you about Less Wrong?
I dislike the discussion/main section divide. I have to check two places for recent comments/posts. Plus, every time I want to make a post I have to decide which section to post it in, and that seems to be a not insignificant mental cost. Actually I can't really tell which posts belong where, so I've ended up posting all of them in discussion "to be safe".
Does checking http://lesswrong.com/r/all/new solve the problem of checking two places?
= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
Can you give more details here? I'm not familiar with extensive-form vs. normal-form games.
Something like that. It seems like the computational concerns are extremely important: after all, a theory of morality should ultimately output actions, and to output actions in the context of a utility function-based model you need to be able to actually calculate probabilities and utilities.
Sure. Say you have to make some decision now, and you will be asked to make a decision later about something else. Your decision later may depend on your decision now as well as part of the world that you don't control, and you may learn new information from the world in the meantime. Then the usual way of rolling all of that up into a single decision now is that you make your current decision as well as a decision about how you would act in the future for all possible changes in the world and possible information gained.
This is vaguely analogous to how you can curry a function of multiple arguments. Taking one argument X and returning (a function of one argument Y that returns Z) is equivalent to taking two arguments X and Y and returning X.
There's potentially a huge computational complexity blowup here, which is why I stressed mathematical equivalence in my posts.