Why do you think there are so many beetles? God is bored. Maybe he created us so we would invent the Wii?
We'll reject the games where everything is too easy. We will not want to live in a universe which is as we wish it were. It'll be unhealthy for us. We'll throw it off. Instead, we'll play games that are just as hard and frustrating as the life we live now.
Then, the robots will discover a way to circumvent thermodynamics and harness us as batteries and we'll all start saying "Woah!". Plus all the time playing computer games with help will make us so stupid that we give superintelligent, super-fast hostile ems a warning like "Dodge this!" before we shoot them because we don't comprehend their abilities. (Perhaps they meant that as a metaphor?)
On the subject of computer games (an underrated area for the study of psychology, economy and even AI, IMO):
During the last 3 years, I have spent just over 1000 hours playing World of Warcraft. Why did I choose to spend (some, incl my wife, might say waste) my time on this? I am fairly wealthy and quite fit - just about any fun activity is open to me. So why do I like WoW? And why 10 million people around the world do the same?
Some important reasons why the game is so pleasurable seem to be:
a) the ultimate goals are pretty clear (so unlike real life...)
b) the "measures of progress" are likewise clear - and there is only one way to go, namely "up"! (again, so unlike real life, apart from possibly "youth" - is that what makes "youth" so good?)
c) the rewards are clear - and are earned so progressively that playing the game seems akin to wireheading (the trickle of XPs, "gold" and new items continuously stimulates some pleasure centre or other).
(Good play involves fairly sophisticated analysis, strategy and tactics, which maintains intetrest... but is beside the point I want to make. The game is attractive to good and poor players.)
With...
@ D. Alex: Some important reasons why the game is so pleasurable seem to be:
a) the ultimate goals are pretty clear (so unlike real life...)
b) the "measures of progress" are likewise clear -
c) the rewards are clear -
This looks like real life without the hard parts. Sure, it makes it more fun, but at the end will you feel rewarded? If you look back now or in a few years to the time spent playing and consider what you could have achieved in real life if you invested the same time into real challenges how will you feel? From my own experience I can...
@D. Alex: (For Robin Hanson: have you heard about the economic studies carried out in WoW setting?)
Since you mentioned economics, did you ever consider the opportunity cost of playing WoW?
What you describe as targets over '4D states' reminds me of Finite and Infinite Games by James Carse. For an example, playing a game of basketball with a winner/loser after an hour of play is a finite game. However, the sport of basketball overall, is an infinite game. So playing a specific video game to reach a score or pass the final level is a finite game, but being a 'gamer' is an infinite game, allowing ever more types of gaming to take place.
Unlike Roland, who is obviously a puritan, I rather enjoy the occasional spot of idleness. For a non-trivial number of people, playing WoW for a couple of hours a day is more fun that playing real life. Rather than make thinly veiled moral judgements about folks for their unproductivity, perhaps he should consider what makes certain games so engaging.
I spent a year playing a lot of WoW, attaining non-trivial sucess in both raiding and competitive PVP, but I gave it away, partly because the time commitment became too great and partly because what passed for progression started to lose its shine.
So, being suitably qualified, I'll take a stab at a few features that make this virtual social experience psychologically rewarding:
Competition with minimised risk. I love fighting. Seriously - nothing beats the adrenaline buzz, time compression and sheer physicality. Unfortunately even controlled fighting in the physical world entails a level of risk that as a father I'm not willing to assume. Simulated violence, while a poor substitute, helps to fill the void.
Persistent progress. Sure online FPS is fun, but when you log on you're always the same guy (more or less). There's also less ris
@ ac: I agree with everything you said except the part about farming a scripted boss for phat lewt in the future. One would think that in the future they could code something more engaging. Have you seen LOTR...
EY: you'll find that it contains some tasks that are - there's no kind word for this - effortful. (I would even say "difficult", with the understanding that we're talking about something that takes 10 minutes, not 10 years.)
Some tasks in WoW can take months to complete, and it's clearly intended by WoW developers. Many tasks require 'raiding', which is an organized, coordinated activity involving up to 40 players, strategy, advance preparations, purchases, crafting etc. -- I have a friend who keeps a calendar of his evening raids and plans his real-world time in advance. When I played WoW, I didn't raid at all because it placed too much constraints on my real-world schedule.
EY: So in the future, we'll have programs that help you play the game
To a certain extent, we already do. Speaking of WoW again, we have Thottbot and Ludwig that help you instantly look up any item or spell (this function isn't build into WoW), talent/glyph calculators, forums where people calculate all these probabilities of critical strikes, and a huge number of addons -- for example Auctioneer, which lets people trade at the Auction House far more effectively (you see mean/median/average prices for ...
MMO of the future lol(some swearing)
And just so I'm not completely off topic, I agree with the original post. There should be games, they should be fun and challenging and require effort and so on. AI's definetly should not do everything for us. A friendly future is a nice place to live in and not a place wher an AI does the living for us so we might as well just curl up in a fetal position and die.
D. Alex:this is a bit out of left field, but: Is a setting like World of Warcraft a good medium for development of AI? Clear goals, clear measures of progress, sufficient complexity to provide an indication of when important insights are achieved, and a safe environment (in the sense that the path to paperclip AI seems unlikely)...?
Ben Goertzel proposes a exactly that (but for different reasons): http://www.kurzweilai.net/meme/frame.html?main=/articles/art0710.html
How about making games that serve a purpose in the real world? Imagine a virtual world that generates and distributes quests and puzzles based on what kind of (robotic) work is needed in the real world. I guess this would go under "removing low-quality work to make way for high-quality work".
http://en.wikipedia.org/wiki/Game_with_a_purpose http://en.wikipedia.org/wiki/Human-based_computation
I watch kittens "playing", definitely building useful skills for the future. I enjoy effort on puzzles and games because each gives me a moment of pleasure on success, and no bad consequences of failure, but some games improve reactions or are otherwise useful.
The "Culture" sequence of novels by Iain M. Banks suggests how people might cope with machines doing all the work. One man works in a cafe, waiting on tables and cleaning up. Yes, the machines could do this work, but he gains happiness from the feeling of serving other people. Oth...
EY: So this is the ultimate end of the prophecy of technological progress - just staring at a screen that says "YOU WIN", forever.
On second thought, playing a modern game IS staring at the "YOU WIN" screen.
Say, you just started playing a game. You did nothing at all, but you're already immortal, you look badass, you have fists the size of a boulder, and you can engage some mean-looking bad guys and win!
So, the actual product of the game industry is 4D "YOU WIN" screens.
There's so much to consider here. For me at least, for something to be fun, I have to know that there's a challenge. For it to be a challenge, there has to be the possibility of failure. There has to be scary parts, that remind you of failure. There has to be multiple real, meaningful, obvious paths that suggest fun in the short term, but disaster in the long term, that you have to look at and reject. There have to be rewards that are enticing, but incredibly rare and difficult, and other rewards that are easier, faster and more localized.
But more im...
Abigail:
"The "Culture" sequence of novels by Iain M. Banks suggests how people might cope with machines doing all the work."
Exactly, I think Culture is highly relevant to most topics discussed here. Obviously, it is just a fictional utopia, but I believe it gives plausible answer to "unlimited power future".
For the reference: http://en.wikipedia.org/wiki/The_Culture
"Since you play the game to get gold and experience points, making the game easier will let you get more gold per unit time: the game will become more fun."
I know this statement is being set up to be knocked down but when I read it I recoiled in disgust. Many people already do not play games with the end goal of increasing the values in the computer that represent gold or experience, because that is a completely empty and pointless thing to care about. My goal when playing games is "to improve myself" and has been for several years. The most fun I have when playing a game is just after I improve my own skill enough to overcome some challenge in a game that I couldn't do before.
I mainly play competitive multiplayer games and extremely difficult singleplayer games. I refuse to play any multiplayer game with meaningful persistent character state because that inevitably makes the game revolve around grinding. Grinding is pointless and stupid and typically does not develop much skill.
"For me at least, for something to be fun, I have to know that there's a challenge. For it to be a challenge, there has to be the possibility of failure."
I go farther than this. For a game to be significantly fun for me I have to ACTUALLY FAIL. Repeatedly. Until I get good.
I find Greg Egan's Permutation City relevant here, particularly the character of Peer. He gives himself arbitrary desires over time, say collecting butterflies or making table legs with a lathe. Re-design the brain to enjoy some arbitrary, meaningless task for a finite time. It is one way of implementing jb's notion. At one point, he constructs a simulation that is perfectly circular: the experience is a closed loop that leads him back to his starting point and mental state. He realizes that he could maintain this one infinitely, or, with no loss, just run it once and permanently shut down.
Or as another character explains, how does it all feel? "However I want it to feel."
what's more fun? a holodeck that you have complete control over? or a holodeck with built in constraints?
playing god might be fun for awhile, but I think everyone would eventually switch over to programs with built in constraints to challenge themselves. the profession of highest prestige will probably people who write really really good holodeck programs.
Computer games are the devil but I agree strongly with Hyphen, the good ones are like sports not work.
"Though, since you never designed your own leg muscles, you are racing using strength that isn't yours. A race between robot cars is a purer contest of their designers."
Eliezer: While people don't design their muscles, they presently don't design their brains either, so a robot car-designing contest seems like just as impure a contest. Even if people did repeatedly redesign their brains, wouldn't this either result in convergence, in which case the contestants would be identical and the contest wouldn't be interesting, or alternatively, the arbitrary initial advantages and disadvantages would just be passed on in modified and perhaps even amplified form and the contest stays as impure as ever. Even if you try to measure the amount of effort the contestants put in, that's no good either because different people are born with unfairly different amounts of will-power.
So what on earth do you mean by "purer contest"?
I didn't say pure, I said purer. If you write your own legs, that's a purer contest than running on muscles you don't understand. If you rewrite your own mind, that's a purer contest than thinking with neural circuits you don't understand. You never made and can never make yourself from scratch, from nothing; but it's possible that the brain you wield could be Truly Part Of You. The implications of which are not exactly straightforward, but I haven't gotten to the section on introspection yet.
What can I say, apart from "Progress Quest"
http://www.progressquest.com/ http://www.progressquest.com/info.php
Officially voted the Top Role Playing Game for Post-Singularity Sentient Beings.
Anonymous.
One common answer to the question "What will we do in the future when we've fixed all that is wrong with today" is "How the hell should I know?"
For example, imagine our neolithic ancestors asking each other the same question. "What will they do in the future when they don't have to worry about food, shelter, or even disease?" I think they could have imagined some things; "They'll make more complicated art." "They'll have more complicated sports."
But I don't they they would have imagined full time mathema...
That one bothered me too. Perhaps you could say bodies are much more peripheral to people's identities than brains, so that in the running case what is being tested is meat that happens to be attached to you and in the robot case it's you yourself. On the other hand I'd still be me with some minor brain upgrades.
Based on the comments here, it would seem that it's the people who reject ultimately-meaningless forms of play - that is, 'play' that doesn't develop skills useful to perpetuation - and concentrate on the "real world" who will end up existing.
And the Luddites will inherit the Earth...
Or, to put it another way:
"Fixing" the future, in a way that renders human beings completely redundant and unnecessary even to themselves, isn't fixing anything. It's creating a problem of unlimited scope.
If that's the ultimate outcome of, say, producing superhuman minds - whether they're somehow enslaved to human preferences or not - then we're trying very hard to create a world in which the only rational treatment of humanity is extinction. Whether imposed from without or from within, voluntarily, is irrelevant.
I don't know about you guys but I'm having fun just trying to keep this rock from rolling back down the hill.
What's most interesting to me is that lizards don't have fun.
Maybe they have fun. But if they do, I'm pretty sure worms don't have fun. A discussion like this one, carried on by lizards (or worms), wouldn't have included the concept "fun".
And if you keep going back in time or down in size, I'm sure you'll find organisms that don't experience pleasure.
Are there other types of possible experiences as qualitatively different and intrinsically good? Are there infinitely many of them? Is charting the course based on "fun theory" like lizards charting the course of the future based on "basking on a hot rock theory"?
Probably. And if the set of organisms that experience pleasure is a proper subset of the set of organisms that experience fun, then the answer is even more likely to be yes.
That should have said "as qualitatively different and intrinsically good as fun?"
"Heaven is being perfect.": Even a circle can't be perfect, in the classical sense of being the best possible circle. Is a circle of 2cm radius better than a circle of 1cm radius? It is much more nonsensical to talk of a person being perfect. It is even more nonsensical to talk of a still-evolving species being perfect.
I didn't mean "proper subset". I mean that if there are organisms that experience pleasure but not fun (or vice-versa), then it's more likely that there's an infinite number of possible "inherently good" noumena like pleasure, fun, and love; and that we've discovered only a small number of them.
And the mapping of attraction to those noumena is entirely subjective! Like in another Elizer Yudkowsky essay: http://lesswrong.com/lw/tn/the_true_prisoners_dilemma/
I'd say that "work" generally consists of activities that are only useful as a means, and not as an end. In general, "work" is anything that you'd rather have someone (or something) else do for you.
For example, you don't like cleaning the toilet, but you want the toilet to be clean, so you clean it anyway. Cleaning the toilet is work.
Anything you wouldn't volunteer to do if you weren't getting paid is work.
So, yeah, let's get rid of work!
@ac:
Unlike Roland, who is obviously a puritan, I rather enjoy the occasional spot of idleness. For a non-trivial number of people, playing WoW for a couple of hours a day is more fun that playing real life. Rather than make thinly veiled moral judgements about folks for their unproductivity, perhaps he should consider what makes certain games so engaging.
I enjoyed playing games myself, so I know what you are talking about.
You mention idleness, which I agree is sometimes worthwhile. This is the package deal fallacy since there are other ways to achieve tha...
caledonian: I agree. if we develop some sort of virtual reality that can provide any desire, we'll just be selecting for people who don't go in and never come out. If so the future will be populated by people who refuse such self gratification.
increasing your social circle(online friends don't count) It will be a small, lonely post-upload life...
On the subject of MMORPGs, I've enjoyed playing one for about a year and then it stopped being fun. In the beginning the interesting part was the world exploration, acts of learning new things, rules and interactions between various parts of the system, feeling of steady advance towards some clearly defined goal and work that was guaranteed to pay off. After a while grinding has started to become annoying and my interest shifted towards minmaxing everything and writing complex scripts to allow bots do the boring parts. Then realisation hit me. In real worl...
Then realisation hit me. In real world an extraordinarily efficient way of doing things is good, it's called an invention. In a game it is called cheating. Nature doesn't care what smart tricks you used to achieve your goals. In a game if it wasn't anticipated by developers it probably counts as an exploit. The universe has a set of unchanging rules, a game is perpetually balanced by series of patches and crutches in unpredictable places. By being creative you are fighting against game developers, which is pointless because they will actively oppose and ge...
EY: I'm not going to touch that one. I'll stick with the much simpler answer of "I wouldn't actually prefer to be a passive experiencer." If I wanted Nirvana, I might try to figure out how to achieve that impossibility. But once you strip away Buddha telling me that Nirvana is the end-all of existence, Nirvana seems rather more like "sounds like good news in the moment of first being told" or "ideological belief in desire" rather than, y'know, something I'd actually want.
Actually, to me Nirvana - or wireheading - sound like...
I don't understand what's the problem Eliezer has with advanced computer games. Why not "waste effort" if it's fun, and all the important work in the real world is anyway getting done?
No, the end of the road for the MMORPGer wouldn't be staring at an "YOU WIN" screen. That's not fun, I want to instead go waste some effort in e.g. advanced ancestor simulations, possibly with my memories of "the future" temporarily erased.
Kaj: But if they were made into orgasmium, even for a short while, then they wouldn't want it to ever stop...
Well, yeah, but most people, if they had their brain removed and replaced by a slab of jelly, wouldn't raise any objections either.
I don't think wireheading is that different. A wireheaded brain is basically a hacked brain that doesn't work any more.
(you could also replace "made into orgasmium" by "given heroin")
Emile: A wireheaded brain is basically a hacked brain that doesn't work any more.
Sure, if you want to define it that way... but I'd be wary of that kind of thinking. Doesn't work for what anymore? It certainly still works for experiencing phenomenal sensations. Doesn't work for the evolutionary purpose of maximizing fitness? Well, we should try to get past that anyhow. Doesn't work for self-preservation? That's just a poorly implemented wirehead. If one were, say, running as code, you could relatively easily have a subprogram that monitored the environment...
"I sometimes think that futuristic ideals phrased in terms of 'getting rid of work' would be better reformulated as 'removing low-quality work to make way for high-quality work'."
Alternatively, you could taboo work and play entirely, speaking instead of various forms of activity, and their various costs and benefits.
...a science fiction story about such a utopia included a conversation between a robot and a person. In the end, the robot said, "But, you can get drunk."
Do we even need the destination? When you consider "fun" as something that comes from a process, from the journey of approaching a goal, then wouldn't it make sense to disentangle the journey and the goal? We shouldn't need the destination in order to make the journey worthwhile. I mean, if the goal were actually important, then surely we'd just get our AI buddies to implement the goal, while I was off doing fun journey stuff.
For a more concrete example:
I like baking fruitcakes. (Something I don't do nearly often enough these days.) Mixing the ra...
I just want to say that your point about valuing actual people and not (potentially illusory) experiences of people is a very important one, and one I wish I could explain better to people who think that maximizing stimulation to the opiate system is the final word on happiness.
Though I've never thought of myself as a Singularitarian, one thing I do look forward to that could be called a "singularity" is the point at which all the problems where winning is what counts have been won---no more war, no more cancer, no more poverty. Amazingly, we are about 80% there, and have risen from roughly 20% in the last few hundred years.
I'm not sure that the difference between 4D states and 3D states is meaningful, with respect to eudaimoniac valuations. Doesn't this overlook the fact that human memories are encoded physically, and are therefore part of the 3D state being looked at? I don't see any meaningful difference between a valuation over a 4D state, and a valuation over a 3D state including memories of the past.
In other words, I can think of no 3D state whose eudaimoniac valuation is worse than that of the 4D state having it as its endpoint.
(In fact, I can think of quite a few whic...
realistic nonsentient
But but anti-zombie principle!
Though, since you never designed your own leg muscles, you are racing using strength that isn't yours. A race between robot cars is a purer contest of their designers.
How do you figure? You didn't design your brain, either, so using your intellect to design a robot car is also using strength that isn't yours.
There's a class of prophecy that runs: "In the Future, machines will do all the work. Everything will be automated. Even labor of the sort we now consider 'intellectual', like engineering, will be done by machines. We can sit back and own the capital. You'll never have to lift a finger, ever again."
But then won't people be bored?
No; they can play computer games—not like our games, of course, but much more advanced and entertaining.
Yet wait! If you buy a modern computer game, you'll find that it contains some tasks that are—there's no kind word for this—effortful. (I would even say "difficult", with the understanding that we're talking about something that takes 10 minutes, not 10 years.)
So in the future, we'll have programs that help you play the game—taking over if you get stuck on the game, or just bored; or so that you can play games that would otherwise be too advanced for you.
But isn't there some wasted effort, here? Why have one programmer working to make the game harder, and another programmer to working to make the game easier? Why not just make the game easier to start with? Since you play the game to get gold and experience points, making the game easier will let you get more gold per unit time: the game will become more fun.
So this is the ultimate end of the prophecy of technological progress—just staring at a screen that says "YOU WIN", forever.
And maybe we'll build a robot that does that, too.
Then what?
The world of machines that do all the work—well, I don't want to say it's "analogous to the Christian Heaven" because it isn't supernatural; it's something that could in principle be realized. Religious analogies are far too easily tossed around as accusations... But, without implying any other similarities, I'll say that it seems analogous in the sense that eternal laziness "sounds like good news" to your present self who still has to work.
And as for playing games, as a substitute—what is a computer game except synthetic work? Isn't there a wasted step here? (And computer games in their present form, considered as work, have various aspects that reduce stress and increase engagement; but they also carry costs in the form of artificiality and isolation.)
I sometimes think that futuristic ideals phrased in terms of "getting rid of work" would be better reformulated as "removing low-quality work to make way for high-quality work".
There's a broad class of goals that aren't suitable as the long-term meaning of life, because you can actually achieve them, and then you're done.
To look at it another way, if we're looking for a suitable long-run meaning of life, we should look for goals that are good to pursue and not just good to satisfy.
Or to phrase that somewhat less paradoxically: We should look for valuations that are over 4D states, rather than 3D states. Valuable ongoing processes, rather than "make the universe have property P and then you're done".
Timothy Ferris is again worth quoting: To find happiness, "the question you should be asking isn't 'What do I want?' or 'What are my goals?' but 'What would excite me?'"
You might say that for a long-run meaning of life, we need games that are fun to play and not just to win.
Mind you—sometimes you do want to win. There are legitimate goals where winning is everything. If you're talking, say, about curing cancer, then the suffering experienced by even a single cancer patient outweighs any fun that you might have in solving their problems. If you work at creating a cancer cure for twenty years through your own efforts, learning new knowledge and new skill, making friends and allies—and then some alien superintelligence offers you a cancer cure on a silver platter for thirty bucks—then you shut up and take it.
But "curing cancer" is a problem of the 3D-predicate sort: you want the no-cancer predicate to go from False in the present to True in the future. The importance of this destination far outweighs the journey; you don't want to go there, you just want to be there. There are many legitimate goals of this sort, but they are not suitable as long-run fun. "Cure cancer!" is a worthwhile activity for us to pursue here and now, but it is not a plausible future goal of galactic civilizations.
Why should this "valuable ongoing process" be a process of trying to do things—why not a process of passive experiencing, like the Buddhist Heaven?
I confess I'm not entirely sure how to set up a "passively experiencing" mind. The human brain was designed to perform various sorts of internal work that add up to an active intelligence; even if you lie down on your bed and exert no particular effort to think, the thoughts that go on through your mind are activities of brain areas that are designed to, you know, solve problems.
How much of the human brain could you eliminate, apart from the pleasure centers, and still keep the subjective experience of pleasure?
I'm not going to touch that one. I'll stick with the much simpler answer of "I wouldn't actually prefer to be a passive experiencer." If I wanted Nirvana, I might try to figure out how to achieve that impossibility. But once you strip away Buddha telling me that Nirvana is the end-all of existence, Nirvana seems rather more like "sounds like good news in the moment of first being told" or "ideological belief in desire" rather than, y'know, something I'd actually want.
The reason I have a mind at all, is that natural selection built me to do things—to solve certain kinds of problems.
"Because it's human nature" is not an explicit justification for anything. There is human nature, which is what we are; and there is humane nature, which is what, being human, we wish we were.
But I don't want to change my nature toward a more passive object—which is a justification. A happy blob is not what, being human, I wish to become.
I earlier argued that many values require both subjective happiness and the external objects of that happiness. That you can legitimately have a utility function that says, "It matters to me whether or not the person I love is a real human being or just a highly realistic nonsentient chatbot, even if I don't know, because that-which-I-value is not my own state of mind, but the external reality." So that you need both the experience of love, and the real lover.
You can similarly have valuable activities that require both real challenge and real effort.
Racing along a track, it matters that the other racers are real, and that you have a real chance to win or lose. (We're not talking about physical determinism here, but whether some external optimization process explicitly chose for you to win the race.)
And it matters that you're racing with your own skill at running and your own willpower, not just pressing a button that says "Win". (Though, since you never designed your own leg muscles, you are racing using strength that isn't yours. A race between robot cars is a purer contest of their designers. There is plenty of room to improve on the human condition.)
And it matters that you, a sentient being, are experiencing it. (Rather than some nonsentient process carrying out a skeleton imitation of the race, trillions of times per second.)
There must be the true effort, the true victory, and the true experience—the journey, the destination and the traveler.