Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

bogdanb comments on Failed Utopia #4-2 - Less Wrong

53 Post author: Eliezer_Yudkowsky 21 January 2009 11:04AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (248)

Sort By: Old

You are viewing a single comment's thread.

Comment author: bogdanb 24 January 2009 02:46:00PM 0 points [-]

Will Pearson: your arguments apply equally well to any planner. Planners have to consider the possible futures and pick the best one (using a form of predicate), and if you give them infinite horizons they may have trouble.

True, whenever you have a planner for a maximizer, it has to decide how to divide its resources between planning and actually executing a plan.

However, your wish needs a satisfier: it needs to find at least one solution that satisfies the predicate "I wouldn't regret it".

The maximizer problem has a "strong" version which translates to "give me the maximum possible in the universe", which is obviously a satisfier problem (i.e., find a solution that satisfies the predicate "is optimal", then implement it). But you can always reformulate these in a "weak" version: "find a way of creating benefit; then use x% resources to find better ways of maximizing benefit, and the rest to implement the best techniques at the moment", with 0 < x < 100 an arbitrary fraction. (Note that the "find better ways part" can change the fraction if it's sure it would improve the final result.)

So, if you just like paperclips and just want a lot of those, you can just run the weak version of the maximizer be done with it: you're certain to get a lot of something as long as it's possible.

But for satisfiability problems, you might just have picked problem that doesn't have a solution. Both "find a future I wouldn't regret" and "make the maximum number of paperclips possible in this Universe" are such satisfiability problems. (I don't know if these problems in particular have a "findable" solution, however, nor how to determine it. The point is that they might be, so it's possible to spend the lifetime of the Universe for nothing.)

The only idea of an equivalent "weak" reformulation would be to say "use X resources (this includes time) to try to find a solution". This doesn't seem as acceptable to me: you might still spend X resources and get zero results. (As opposed to the "weak" maximizer, where you still get something as long as it's possible.) But maybe that's just because I don't care about paperclips that much, I don't know.)

* * *

Now, if you absolutely want to satisfy a predicate, you just don't have any alternative to spending all your resources on that. OK. But are you sure that "no regrets" is an absolutely necessary condition on the future? Actually, are you sure enough of that that you'd be willing to give up everything for the unknown chance of getting it?