= 27d567bc2d48d9f40e9ccc20ea370b15)
Wei_Dai comments on That Magical Click - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (400)
When you play an engaging computer game, does it detract from your experience knowing that all the tasks you are performing are only there for your pleasure, and that the developers could have easily just made you click an "I Win" button without requiring you to do anything else?
I suspect that status effects might be important here. When we play a video game, we choose to do it voluntarily, and so the developers are providing us a service. But if the universe is controlled by an AI, and we have no choice but to play games that it provides us, then it would feel more like being a pet.
The AI could also try to take that into account, I suppose, but I'm not sure what it could do to alleviate the problem without lying to us.
If you think of FAI as Physical Laws 2.0, this particular worry goes away (for me, at least). Everything you do is real within FAI, and free will works the same way it does in any other deterministic physics: only you determine your decisions, within the system.
It's not quite the same, because when the FAI decided what Physical Laws 2.0 ought to be, it must have made a prediction of what my decisions would be under the laws that it considered. So when I make my decisions, I'm really making decisions for two agents: the real me, and the one in FAI's prediction process. For example, if Physical Laws 2.0 appears to allow me to murder someone, it must be that the FAI predicted that I wouldn't murder anyone, and if I did decide to murder someone, the likely logical consequence of that decision is that the FAI would have picked a different set of Physical Laws 2.0.
It seems to me that free will works rather differently... sort of like you're in a Newcomb's Problem that never ends.
It just means that you were mistaken and PL2.0 doesn't actually allow you to murder. It's physically (rather, magically, since laws are no longer simple) impossible. This event has been prohibited.
I would expect that an FAI would not force us to play games, but would make games available for us to choose to play.
It's not that an FAI would force us to play games, but rather there's nothing else to do. All the real problems would have been solved already.
That's not necessarily true. We might still have to build a sturdy bridge to cross a river, it's just that nobody dies if we mess up.
Likewise, if one's mind is too advanced for bridge building to not be boring, then there will be other more complex organizations we would want, which the FAI is under no obligation to hand us.
I think we can have a huge set of real problems to solve, even after FAI solves all the needed ones.
How is bridge-building not a game when the FAI could just flick a switch and transport you across the river in any number of ways that are much more efficient? When you're building a bridge in that situation, you're not solving the problem of crossing a river, you're just using up resources in order to not be bored.
Because it refuses to do so?
If you're 16 and your parents refuse to buy something for you (that they could afford without too much trouble) and instead make you go out and earn the money to buy it yourself, was solving the problem of how to get the money "just a game"?
Yes, if the parents will always be there to take care of you.
We can wirehead children now.
We want them to be more than that.
The only reason we want that is that civilization would collapse without anyone to bear it. If FAI bears it, there is no pressure on anyone.
It can't actually do that, because it's not what its preference tells it to do. The same way you can't jump out of the window given you are not suicidal.
By that reasoning, World of Warcraft is not a game because the admins can't make me level 80 on day 1, because that's not what their preferences tell them to do... Or am I missing your point?
I'm attacking a specific argument that "FAI could just flick a switch". Whether it moves your conclusion about the described situation being a game depends on how genuine your argument for it being a game was and on how much you accept my counter-argument.
Could one of you précis the disagreement in a little more detail and with background? When you and Wei Dai disagree, I'd really like to understand the discussion better, but the discussion it sprang out of doesn't seem all that enlightening - thanks!
When I build a bridge in a game, I get an in-game reward. I don't get easier transport to anywhere. If I neglect to build the bridge or play the game at all, I still get to use all the bridges otherwise available to me. 'Real' bridges are at the top level of reality available to me. Even the simulation hypothesis does not make these bridges a game.
Why do I want to cross the bridge? To not be bored, to find my love, or to meet some other human value. The AI could do that for me too, and cut out the need for transport. If we follow that logic even a short way, it would be obvious that we don't want the AI doing certain things for us. If there is danger of us being harmed because the FAI could help but won't it need merely help a little more, getting closer to those things we want to do ourselves. If we're in danger of being harmed by our own laziness, it need only back off. (It might do this at the level of the entires species, for all time, so individuals might be bored or angry or not cross rivers as soon as they would like, but it might optimize for everybody moment to moment.)
If there are things we couldn't stand to have a machine do, and couldn't stand for it to not help us with, I think those would be incoherent volitions.
One way I imagine that would work for me is if the AI explained with sufficient persuasion that there simply isn't anything more meaningful for me to do than to play games. If there actually is something more meaningful for people to do, then the AI should probably let people do that.
An AI could persuade you to become a kangaroo -- this is a broken criterion for decision-making.
I am skeptical that rationality and exponentially greater-than-human intelligence actually confers this power.
It doesn't matter if it does or not; the fact that you can conceive of situations where persuadability would fail as a criterion immediately means it fails.
Well, that was the big controversy over the AI Box experiments, so no need to rehash all that here.
This is a category error. Meaningfulness is in your mind and in intersubjective constructions, not in the objective world. There is no fact of the matter for the AI to explain to you.