My claim is that Altman can't do it alone, he needs the cooperation of at least a fraction of the existing system (the government+business leaders who form the Oversight Committee - some of whom might be the biggest OpenAI shareholders). Once you get enough of the existing system involved, it becomes plausible that they keep money around for some of the same reasons that the existing system currently keeps money around. Near the end of the Oversight Committee ending, it says:
... (read more)As the stock market balloons, anyone who had the right kind of AI investments pulls further away from the rest of society. Many people become billionaires; billionaires become trillionaires. Wealth inequality skyrockets.
I'm not a lawyer, but if it were secret, and done along with the alignment team, and had a chance of working, then wouldn't it qualify as conspiracy to commit treason?
If not, then as long as it negatively affects residents of the state of California, it qualifies as misrepresenting the capacity of an AI to cause catastrophic harm to property, punishable by a fine of up to $100,000 under SB 53!
They aren't aligned in this way. If they were, they wouldn't try to cheat at programming tasks, much less any of the other shenanigans they've been up to. These may seem minor, but they show that the "alignment" hasn't actually been internalized, which means it won't generalize.
Sorry, I didn't mean to make a strong claim that they were currently 100% aligned in this way, just that currently, insofar as they're aligned, it's in this way - and in the future, if we survive, it may be because people continue attempting to align them in this way, but succeed. There's currently no form of alignment that fully generalizes, but conditional on us surviving,... (read 509 more words →)
I agree it's not obvious that something like property rights will survive, but I'll defend considering it as one of many possible scenarios.
If AI is misaligned, obviously nobody gets anything.
If AI is aligned, you seem to expect that to be some kind of alignment to the moral good, which "genuinely has humanity's interests at heart", so much so that it redistributes all wealth. This is possible - but it's very hard, not what current mainstream alignment research is working on, and companies have no reason to switch to this new paradigm.
I think there's also a strong possibility that AI will be aligned in the same sense it's currently aligned - it follows... (read more)
You might want to try adapting some of the ones from http://slatestarcodex.com/2018/02/06/predictions-for-2018/ and the lists linked at the bottom.
Agreed that some people were awful, but I still think this problem applies.
If somebody says "There's a 80% chance of rain today, you idiot, and everyone who thinks otherwise deserves to die", then it's still not clear that a sunny day has proven them wrong. Or rather, they were always wrong to be a jerk, but a single run of the experiment doesn't do much to prove they were wronger than we already believed.
The weatherman who predicts a 20% chance of rain on a sunny day isn't necessarily wrong. Even the weatherman who predicts 80% chance of rain on a sunny day isn't *necessarily* wrong.
If there's a norm of shaming critics who predict very bad outcomes, of the sort "20% chance this leads to disaster", then after shaming them the first four times their prediction fails to come true, they're not going to mention it the fifth time, and then nobody will be ready for the disaster.
I don't know exactly how to square this with the genuine beneficial effects of making people have skin in the game for their predictions, except maybe for everyone to be more formal about it and have institutions that manage this sort of thing in an iterated way using good math. That's why I'm glad you were willing to bet me about this, though I don't know how to solve the general case.
Re: the "when friends and colleagues first come across this conclusion..." quote:
A world where everybody's true desire is to rest in bed as much as possible, but where they grudgingly take the actions needed to stay alive and maintain homeostasis, seems both very imaginable, and also very different from what we observe.
I think it says something good about our community that whoever implemented this feature assumed people would be more likely to want to write mathematics than to discuss amounts of money.
Wei Dai, one of the first people Satoshi Nakamoto contacted about Bitcoin, was a frequent Less Wrong contributor. So was Hal Finney, the first person besides Satoshi to make a Bitcoin transaction.
The first mention of Bitcoin on Less Wrong, a post called Making Money With Bitcoin, was in early 2011 - when it was worth 91 cents. Gwern predicted that it could someday be worth "upwards of $10,000 a bitcoin". He also quoted Moldbug, who advised that:
If Bitcoin becomes the new global monetary system, one bitcoin purchased today (for 90 cents, last time I checked) will make you a very wealthy individual...Even if the probability of Bitcoin succeeding is epsilon, a million... (read 986 more words →)
I don't know what you're trying to say here. Some set of important people write the spec. Then the alignment team RLHFs the models to follow the spec. If we imagine this process continuing, then either:
I'm claiming either of those options is hard.
(I do think in the future there will may some kind of automated pipeline, such that someone feeds the spec to the AIs, and some other AIs take care of the process of aligning the new AIs to it, but that just regresses the problem.)