The 5 and 10 problem is basically what happens when your agent asks "what are the logical implications if 5 is chosen?" rather than "If we do causal surgery such that 5 is chosen, what's the utility?"
There are other ways to avoid the 5 and 10 problem, but I think they're less general than using causality.
If it's worth saying, but not worth its own post (even in Discussion), then it goes here.
Thread started before the end of the last thread to ecourage Monday as first day.