I'm struck by the vastly incompatible views of the future he seems to be holding simultaneously. At the beginning he says, "economy will double every two weeks," and "if you save a little the investment money will make you rich from the economy doubling" coupled with "the entire EM economy could be located all in one small Scandinavian country near where uploading is first created," and "the population 10 trillion cities will consume huge amounts of resources, destroy the environment, and eventually consume the globe." How does that count as anything except a hard takeoff that leaves humanity in the dust, and how do you plan to cash out your investment from the immortal Em society when they decide not to honor your 10^26% compound interest?
I was left with the impression that a lot of the ideas were made because he wanted them to be real first, then were rationalized how it would work later. Some of the things discussed seem especially incoherent. His whole section on Em socialization seems odd when you can just duplicate friends at any time. In fact, the entire talk seems predicated entirely on having near-perfect mind theft security which is covered at the very end in a single slide.
There's also a bit about corporations at ~37 mins. A company with 256 line workers that organizes in 4 man teams needs with 64 line bosses, 16 higher bosses, 4 VPs, and one executive. That's 5 levels of communication which can cause all sorts of problems, not to mention the cost in ~25% of the company being managers. The solution he describes is a single, time accelerated boss running at ~21x speed able to coordinate all 64 teams and keeping the entire company running as one entity. (Note that uber-boss is still taking up 25% of the company's em time.)
The issue I have with this is that it ignores the entire advantages of emulation discussed earlier in the talk, namely duplication of workers. If you can duplicate workers effectively infinitely (eg plumber example) and you can create and destroy workers every 10 minutes (eg computer programmer example), then you don't need an organization at all. Parallelizable tasks can all be solved whenever (again, plumber), and all the work that remains will be in serial. Ems will be tasked with jobs on their instantiation, and will return their results to whoever needed it when their task is completed (which could just be another worker). This brings corporations outside of the realm of bosses and economists and pulls them into the realm of computer scientists. Corporations will look a lot less like pyramids and a lot more like computer programs.
I feel like I'm gonna lose some karma on this comment, but... yeah. Hanson normally strikes me as very insightful but this talk just felt like a mess.
Oh god, running an Em could just be a function call. Guess I'm not sleeping tonight.
Forking processes in Unix make use of copy-on-write memory, and forking uploads could perhaps be like that, so that when an upload copy terminates it literally is just losing some new memories. Would this be better or worse, I wonder, than making and deleting complete copies?
It might be worse for example if you are a total utilitarian and complete copies count more than copy-on-write forks.
And in case anyone is wondering, it might be better because terminating a copy-on-write fork seems less like killing someone than deleting a complete copy.
how do you plan to cash out your investment from the immortal Em society when they decide not to honor your 10^26% compound interest?
Why would you assume investments won't be honored?
As Hanson points out, we could rob our retirees of their investments. This doesn't actually happen because "non-retiree" is a shallow category that doesn't make for an actual functioning political/military coalition. Note that right-handed people are a large majority and could form a political coalition to rob left-handed people of their money. Yet this hasn't actually happened.
He was talking about Ems at the equivalent of .3mm space living 500 years in a month. If you invest when we start seeing a 1 month doubling period and cash out two weeks later, then that's longer than America's been a country. Two months later goes back to before the dark ages, four months is the death of Jesus, and holding on to your investment for a year is basically the entirety of recorded civilization. If you had a time machine and went back and invested in any of those time periods, what are the chances you'd get that money back?
Hanson's analogy about foragers seems more apt. Something along the lines of "there are foragers still today, we didn't kill them off... as long as they're using something we don't care about we're not that bothered by them." Just remember what happens any time native foragers 1) have something we want or 2) try to make a claim against anything we have. It never ends well for the foragers.
Well, you completely ignored the point I made, which is that there is no reason to think emulations would conspire with each other to deny people their investments. Many emulations would also have long periods of investing and money and earning enormous amounts through interest payments. Many non-uploaded humans would probably invest money in funds that would be run (like almost anything else) by uploaded people. Both rich uploads and upload fund managers would have an incentive to oppose confiscations.
If you had a time machine and went back and invested in any of those time periods, what are the chances you'd get that money back?
Benjamin Franklin actually did invest money in 1790, and the amount with 200 years of interest really was paid out in the way specified in the trust. People have owned land titles and shares in businesses for over a thousand years through multiple dynasties and regimes.
1) have something we want or 2) try to make a claim against anything we have. It never ends well for the foragers.
The difference being that foragers don't have titles/deeds or even a clear understanding of property-ownership.
ETA: I should really proofread my comments before I post.
The difference being that foragers don't have titles/deeds or even a clear understanding of property-ownership.
So what? History is still not kind. I pointed out 2 more examples to Hanson, which I think he blogged about, where someone doing something very similar to Franklin saw the money completely wasted. 1/3 odds is not especially encouraging given that the analysis has only just begun...
For example, a deeper analysis might include the Islamic world's perpetual charity corporation, the waqf which form dates back almost to the beginning, collectively comprised a huge chunk of various economies - but while they did much better than one might expect, most still ceased to exist at some point. As far as I know, every wafq started in, say, 1000 AD is now dead. There may be a few Egyptian survivors from later centuries, attached to the University, but nevertheless, the overall picture is one of a very low survival rate. (I've pointed out waqfs to Hanson multiple times, but I don't know what he makes of them.)
Just a rough dirty outline since I'm headed out of town for two days. I may make a discussion post about some of the points later.
The natural human method of property division is killing people and taking their stuff. Capitalism is a blip on the radar.
Ems will be entering an entirely different context than 20th century humans. We'll be lucky if capitalism translates to the digital world at all.
Ems vs human will be a natural division point. Every human (if lucky) becomes a retiree at some point, no Ems become human. Humans do not have a good track record of respecting the property rights of those different to them.
Humans literally cannot object to Ems doing this. We literally cannot think fast enough to stop it from happening and it only needs happen once in 5000 years each year for us.
5000 years? That's older than the entire concept of capital. Unless capitalism is the one true economic system for all eternity, you'll be out of luck. Historically, the vast majority of economic systems wouldn't honor your holding onto that property without a means to personally enforce it.
5000x wasn't the cap. Hanson made a comment about potential 1,000,000x speeds. One year goes longer back than our ancestors could even say the word 'capitalism'. I say ancestors because homo sapiens weren't around back then. I'm imagining making a capital investment to Krogg, and trying to get that repaid today.
This ignores the possibility of Em-space becoming a free for all of replicators trying to ascend into Godhood. I consider it the most likely possibility (but that's the subject of another thread). But even if that doesn't happen and they keep capitalism, and Ems have no government shakeups, and they bother keeping track of debts to humans, you won't be able to withdraw because...
Sheer magnitude of value. A human working earning, lets say $20,000 in one year pre-uploading, is worth one man-year of work. A human investing that money and earning a year's interest during the 1-month doubling time frame suddenly has $81,000,000. People would happily steal from retirees if that much money was changing hands.
Meanwhile the average wage has dropped to $4 a year. There is no moral system I can imagine that figures it's okay for one person's year of work to be worth someone else's 20-million years of work just because they got lucky when they made it.
Scratch that. I know that there's a similar-ish disparity between a CEO and a bottom quintile Nigerian. Lucky for the CEO, current Nigerians don't run at 5000x speed because they would happily take his money away and be morally okay with that. Lets say some Ems are on the side of giving humans back their money. You're going to have literally a thousand times the current population of the earth as poor Nigerians who will be set for a literal million years if they can get just a fraction of the money away from the old human dinosaurs who literally cannot object to them. That's a lot of literals.
Similarly, the opportunity cost of 'pay off that human' vs 'give 10,000,000 ems a better standard of living' is an easy choice. Who could morally side with the human? Well, maybe a human but they won't be making that decision.
All the goods are digital. You can't eat zeros and ones. Your money will increase by 4,000x+ a year, but the number of apples won't increase by 4,000x+ a year, so the gains will just be inflated away for physical goods.
Actually, the economy will probably become mostly decoupled between digital and physical goods. Ems may increase thinking capacity a trillion fold, but it's not they'll will significantly increase the number of apples we grow.
Scratch that. Having Ems will increase apples, because all the computer programmers, and bureaucrats, and entertainers, and etc, etc, etc, will now be out of jobs except for growing apples and delivering microchips to the Em towers. So, I guess that's nice?
My original post was expressing derision at the idea that you could live off the interest from Ems. I understand that it's not a popular message to say, "Ems are gonna hard takeoff, and leave us in the dust, but we won't necessarily go extinct (unless we do)." However, it's a lot more honest than promising fantastic wealth just for surviving until uploading even if you yourself don't upload. As I mentioned above these are rough objections, but I may make a post.
One key objection to this argument is that even though a human with enough money to survive is fantastically rich relative to individual digital folk (and immensely worth expropriating from the perspective of individuals, or those in physical proximity), she can still be vanishingly poor relative to the em society as a whole. If 1 in 1,000,000,000,000,000 available units of free energy within a certain jurisdiction goes to humans while the rest goes to much cheaper digital life, national-scale digital politicians won't be able to promise any noticeable improvement in average living standards in exchange for expropriating the humans.
You seem to be under the impression that I am promising a guarantee that every form of retiree investment will live up to all retiree hopes, no matter how much change happens in the political or economic systems. But even in times of peace where law is upheld, many supposedly "safe" investments will disappoint. The recent downturn is an obvious example. In such times of turmoil, even government guaranteed investments may fail. And in times of war, there are even fewer safe investments.
But this is all a pretty generic fact about the future - the more change it sees, the more at risk are individual investments. If change speeds up, then the realization of this risk speeds up. There really isn't a realistic alternative -- all of this applies no matter what kids of big changes are coming.
It periodically does happen that governments default on their pensions, either completely or partly. Consider Argentina, or Greece. Assorted California cities are currently in bankruptcy court asking a judge to let them reduce their pension obligations.
Even beyond government default, private-sector Investments routinely go bust. If your pension was run by a bank that turned out to be or to become crooked, your long-term compound interest may never happen.
I'm not talking about government run pensions, but about private retirement accounts or other investments owned by retirees.
There again, there is tail risk. You may not be made whole if the pension custodian is bankrupted by fraud, seized by the government, or otherwise subject to accident.
There's an easy empirical way to assess this. Ask how common are violent political revolutions and what happens to financial products sold before the revolution. Did the All-Russia Insurance Company continue to pay out on policies after the 1917 Revolution? The headquarters building was seized by the Cheka and I believe the owners and executives were exiled or liquidated. What happened if you had your pension investments in Iran in 1979?
Yes, there are rich, developed, stable countries where revolution is rare. How long will they stay that way? A century? Two centuries? In the presence of Singularity-scale disruption? These stories about multiplying wealth by factors of thousands or more seem to be making a very heavy bet on permanent stability of the sort never previously seen on Earth.
Parallelizable tasks can all be solved whenever (again, plumber), and all the work that remains will be in serial. Ems will be tasked with jobs on their instantiation, and will return their results to whoever needed it when their task is completed (which could just be another worker). This brings corporations outside of the realm of bosses and economists and pulls them into the realm of computer scientists. Corporations will look a lot less like pyramids and a lot more like computer programs.
This doesn't make sense to me as an objection. Some tasks are not very short and parallelizable, so you need extended serial work. And some of those tasks will profit from coordination between different minds with different skills, or different memories from earlier in the project. Coordinating those units would require some kind of communication and management.
I'm sure it says more about my imagination than the nature of future work, but I still can't imagine anything one can meaningfully work on that can't be either parallelized, or treated like a computer program. It makes it hard to think about the subject without a concrete example to wrap my head around.
Take 50 researchers working on related topics to improve an algorithm to do object recognition in industrial robots. Each topic involves lemmas building upon lemmas, experiments selected based on previous experiments, and other serial steps. The steps are different for the different subtopics. As intermediate results come in, communication and management are used to allocate resources between subtopics, to determine which researchers spend how much time explaining their work to one another to cross-fertilize insights, and so forth.
Notes:
- So he's writing a book on em economics? I'm actually a little disappointed - I was hoping for a grand synthesis of all the health care/Near-Far/signaling material on OB. Well, hopefully he'll be able to marshal all sorts of good details and analogies that he has to skim over in a talk (eg. when he talked about the big economy being spatially small, I thought of the Kuznets curve; location made me think of datacenters, which fortunately he did bring up)
- An Apple user. Always kind of thought of him as a PC.
- The transition example is interesting. I didn't realize that there would be several years before world growth hits insane numbers like 100+% annual growth. This undermines some of my ideas about investing for an em scenario: instead of investing now heavily in equities, it may be better to instead wait for world growth to hit 7%+ and then invest as frantically as possible.
- Luxurious virtual environments will be common: this seems trivially false or at least questionable. Just previously, he points out that wages and costs will fall to the level at which another em can be created/copied. Even if a luxurious virtual environment were cheap in some absolute sense, that doesn't say they'll be cheap in a relative sense. Existing virtual environments require specialized hardwired mini-supercomputers (GPUs) to handle rendering environments, much of which is static, and fall far short of 'luxury'. (As nice as any existing video game like Crysis may look, would you not find being imprisoned in it very unpleasant, to say nothing of extremely luxurious?) AAA games are in a well-known death spiral as the gigabytes of artwork and other demands inflate production costs into the hundreds of millions already. I'd find this point much more convincing if he instead took a tack more like 'yes, it will require terabytes of artwork and gigaflops of rendering, but this will only be 1% or less of the most plausible estimates for emulating minds, and so it's quite plausible such environments will yield a +1% productivity improvement' (until selection pressures eliminate the need or desire for luxurious environments because that's 1% that could be spent better, anyway...).
- It was notably light on the issues of values (farmer vs forager) and selection pressure leading to em hell.
- It was much longer and more comprehensive than I thought. It stands as a pretty decent summary of all the em posts on OB.
Maybe I can write more on near-far and signaling in another book. One thing at a time. Most of the things that make our physical world luxurious or impoverished have little to do with the cost of simulating them. A dirty smelly hut is just as expensive to simulate as a vast mansion. Yes, they might spend 0.1% more relative to brain computing costs on computing VR if that increases work productivity by more than 0.1%.
The point: Not that a vast mansion will be too difficult to simulate relative to a smelly hut, but that anything simulated in sufficient detail will become luxury, while non-wealthy ems live in the relative squalor of palatial mansions that just don't look/feel/sound/smell quite right, in an unpleasant way.
Yes, anything is expensive to sim in very high detail. But it isn't at all clear that in typical settings the amount of detail you can get for say 0.1% of the cost of running a brain is usually unpleasant or disturbing.
You're right, my intuitions had been swayed toward the "expensive verisimilitude" direction by the part in the beginning of Permutation City where the EMs keep committing suicide.
A dirty smelly hut is just as expensive to simulate as a vast mansion.
A mansion will have dozens or hundreds of rooms; how can equally difficult rooms be just as expensive to simulate when there are hundreds more rooms in one scenario than another?
If the em is only in one room at a time, only one room must be simulated in detail at any one time.
The future will have little winding passages constructed to prevent line-of-sight from one room to another?
Deus Ex was righter than they knew.
The rooms will still need to be designed and available somewhere; even AAA games can't get away with indefinitely big canned environments no matter how many paging or zoning tricks they use to reduce the immediate rendering costs.
So he's writing a book on em economics? I'm actually a little disappointed - I was hoping for a grand synthesis of all the health care/Near-Far/signaling material on OB.
Em economics is a probably-irrelevant idea, since engineered machine intelligence is likely to continue beating it. It's one of Hanson's long shots.
Robin talks about a class system where those running at higher speeds are in a higher social class. But given Robin's assumption that cost for emulation speed will be linear up to 1000000x, I don't see why anyone (except those with physical work, which Robin estimates at 20% of the population) wouldn't want to run at the top end of this range. In his scenario of a boss running at 21x the speed of the workers, why isn't the whole team being run at the higher speed? Does anyone understand his reasoning here?
In order for faster ems to talk to each other naturally, they have to be closer to each other, and thus occupy more expensive prime real estate. So they don't want to be faster than they need to be to match the other tasks with which they coordinate.
That doesn't make intuitive sense to me. Surely even fast ems living in cheap real estate will still have plenty (millions? billions?) of people to talk to in real time, even if they have fewer people to talk to compared to those living in prime real estate? Given that running slower has significant costs (you have to pay the same storage costs as faster ems but can do less work, not being able to accumulate experience/knowledge as fast as others and losing competitive edge as a result) I don't see how it's worth those costs just to have more potential people to talk to.
Also, if you're assuming that it's fairly cheap to speed up and slow down emulations, which you apparently are, why don't they run at a fast speed normally and only slow down when they need to talk to distant others with low subjective latency, which may be pretty rare?
For any given task there will be particular people you need to talk to to get it done. I expect hardware would be specialized for particular speeds, but that minds could be moved between hardware of differing speeds in order to change speeds. In general most tasks have a particular time they need to be done, with only minor rewards for doing it much sooner.
In his scenario of a boss running at 21x the speed of the workers, why isn't the whole team being run at the higher speed? Does anyone understand his reasoning here?
How fast you run these employees depends on the economics of your industry, I think the idea is that coordination failure is expensive. Thus if running bosses faster than workers avoids such failures it seems more justified running bosses faster than nearly any other kind of worker. The value of good management in a large company is much higher than the productivity boost any one low level worker could acheive. He touches this when he notes that it is vital the most competent people are as high up the chain as possible.
Sorry, I think I didn't explain well enough why it doesn't make sense to me, so let me try again. In his example there are 256 workers and 64 line bosses running at 1x, and a CEO running at 21x. Why not instead have 16 workers, 4 line bosses, 1 CEO, all running at 16x, which would do the same amount of work in the same amount of time? If we assume that 21x is the maximum feasible emulation speed, it doesn't seem plausible that slowing down the workers to 1x saves enough money (compared to running them at 16x) to make up for increasing the memory requirement by 16 times.
Theoretically you'd be running each of the 256 workers at 1,000,000x speed already. The boss goes into 21,000,000x speed, but has to pay a non-linear cost for that so you can only have one person at that speed. It would require a very particular price/speed discrimination structure to make that viable though.
The other option is that you're in a job that needs 256 different skill sets and we haven't learned how to swap out parts of people's personality yet. Eg, you're translating a book into 256 languages and each person only knows 1 language.
Although neither scenario strikes me as particularly likely.
In his example there are 256 workers and 64 line bosses running at 1x, and a CEO running at 21x. Why not instead have 16 workers, 4 line bosses, 1 CEO, all running at 16x, which would do the same amount of work in the same amount of time?
Evidence suggests that coordination is hard.
That 16 workers running at 16x overseen by 4 line bosses and 1 CEO will suffer more coordination failure that can be avoided by running 256 workers at 1x with one CEO running at 21x, seems plausible and even likely if you assume these are human-like minds.
That 16 workers running at 16x overseen by 4 line bosses and 1 CEO will suffer more coordination failure that can be avoided by running 256 workers at 1x with one CEO running at 21x, seems plausible and even likely if you assume these are human-like minds.
Robin's scenario is 256 workers and 64 line bosses running at 1x plus one CEO running at 21x who directly oversees the line bosses not the workers (you can check for yourself here). This offers no coordination advantages over having 16 workers, 4 line bosses and one CEO all running at 16x, as far as I can tell.
I'm not sure about that. What I've seen of the management literature suggests that the complexity of coordination and oversight problems is strongly nonlinear on the number of workers overseen, and clocking faster would produce only linear improvements. It might still make sense to run the CEO at a faster clockspeed, since that role has to deal with additional coordination problems that aren't entirely under the company's control, but this line of thought suggests to me that smaller numbers of more individually productive workers would be more efficient overall than maximizing the workforce size.
Robin may have been assuming abundant memory and scarce CPU time? I agree though that unless memory costs are very low this is a problem in the examples.
Robin may have been assuming abundant memory and scarce CPU time?
He's not saving on CPU time (i.e., total number of instructions executed), but substituting more, slower processors for fewer, faster processors and also using more memory. We don't see a lot of this today. For example render farms and data centers all use essentially the fastest CPUs available. Some operations might back off a few notches from the bleeding edge in order to save money, but it's not even close to 2x much less 21x. My earlier "doesn't seem plausible" may be too strong, but I don't understand why Robin seems to be predicting this as the most likely scenario. If he has some specific reasons why the economics will likely work out this way, I'd very much like to see it.
We don't see a lot of this today...it's not even close to 2x much less 21x.
We see plenty of this today. Every processor you use with multiple slower cores rather than a single core screaming at 4ghz is making the slow parallel vs fast serial tradeoff. Processor migration and power-saving modes are other examples where the tradeoff is made dynamically ARM processors are hugely abundant in embedded and mobile spaces, and the ARM design is an example of trading off CPU time for other things like reduced transistor count or (especially) power consumption. ARM or Atom chips are making inroads into datacenters because power consumption & cooling are becoming such issues, and we can expect parallelisation to continue for power saving.
Hmm, apparently my knowledge of server hardware was a bit outdated. ARM processors being used by data centers are running at about 1.5 ghz, and it looks like with extreme overclocking it's possible to push x86 processors up to 8 ghz which gives a factor of about 5x. So probably there will be some significant difference between the fastest and slowest uploads, and 21x may not be totally implausible.
One benefit of running on a lower speed is that you can interact with things farther away from you while it still seems instantaneous. although i have no idea why that would be more important for the workers than for the boss.
Found this talk worth watching. A transcript would be great, as would getting the slides. Perhaps RH will post a link to this on Overcoming Bias soon?
Thank you! I should have checked the date of this talk instead of assuming it is new. Quite a bit of lag, LW needs to read more OB. (^_^)
Better suited to the open thread
Edit: Or as you suggest here, a "Lecture/Talks/Discussions" thread. I appreciate the link to the talk. (I'm watching it right now, and wish it had been available a little while ago when I was writing a paper on WBE economics.) But it's not suited for it's own discussion thread unless you have some analysis/argument to supply. Or even a transcript of the talk.
Thanks! I didn't actually know about the open thread but felt that it might not be best to post in the discusion. Still think a lecture thread would be the best place for posts like this one.
He's lecturing on the future of evolution - without mentioning evolutionary theory. Economics evidently needs more Darwin.
Lecture at youtube.
Sorry - haven't watched it yet so no summary, but I expect it to be fun.