True, in a way. Without solving said mystery (of how an animal brain produces not only calculations but also experiences) you could theoretically create philosophical zombie uploads. But in this post what is really desired is to save all conscious beings from death and disease by uploading them, so to that effect (the most important one) it still looks impossible.
(I deleted my post because on hindsight it sounded a bit off topic.)
I never linked complexity to absolute certainty of something being sentient or not, only to pretty good likelihood. The complexity of any known calculation+experience machine (most animals, from insect above) is undeniably way more than that of any current Turing machine. Therefore it's reasonable to assume that consciousness demands a lot of complexity, certainly much more than that of a current language model. To generate experience is fundamentally different than to generate only calculations. Yes, this is an opinion, not a fact. But so is your claim!
I ...
"There is no reason to think architecture is relevant to sentience, and many philosophical reasons to think it's not (much like pain receptors aren't necessary to feel pain, etc.)."
That's just non-sense. A machine that makes only calculations, like a pocket calculator, is fundamentally different in arquitecture from one that does calculations and generates experiences. All sentient machines that we know have the same basic arquitecture. All non-sentient calculation machines also have the same basic arquitecture. The likelihood that sentience will arise in ...
There's just no good reason to assume that LaMDA is sentient. Arquitecture is everything, and its arquitecture is just the same as other similar models: it predicts the most likely next word (if I recall correctly). Being sentient involves way more complexity than that, even something as simple as an insect. It claiming that it is sentient might just be that it was mischievously programmed that way, or it just found it was the most likely succession of words. I've seen other language models and chatbots claim they were sentient too, though perhaps ironical...
Right then, but my original claim still stands: your main point is, in fact, that it is hard to destroy the world. Like I've explained, this doesn't make any sense (hacking into nuclear codes). If we create an AI better than us at code, I don't have any doubts that it CAN easily do it, if it WANTS. My only doubt is whether it will want it or not. Not whether it will be capable, because like I said, even a very good human hacker in the future could be capable.
At least the type of AGI that I fear is one capable of Recursive Self-Improvement, which will unavo...
The post is clearly saying "it will take longer than days/weeks/months SO THAT we will likely have time to react". Both are highly unlikely. It wouldn't take a proper AGI weeks or months to hack into the nuclear codes of a big power, it would take days or even hours. That gives us no time to react. But the question here isn't even about time. It's about something MORE intelligent than us which WILL overpower us if it wants, be it on 1st or 100th try (nothing guarantees we can turn it off after the first failed strike).
Am I extremely sure that an unaligned ...
Your argument boils down to "destroying the world isn't easy". Do you seriously believe this? All it takes is to hack into the codes of one single big nuclear power, thereby triggering mutually assured destruction, thereby triggering nuclear winter and effectively killing us all with radiation over time.
In fact you don't need AGI to destroy the world. You only need a really good hacker, or a really bad president. In fact we've been close about a dozen times, so I hear. If Stanislav Petrov had listened to the computer in 1983 who indicated 100% probability ...
"You may notice that the whole argument is based on "it might be impossible". I agree that it can be the case. But I don't see how it's more likely than "it might be possible"."
I never said anything to the contrary. Are we allowed to discuss things that we're not sure whether it "might be possible" or not? It seems that you're against this.
Tomorrow people matter, in terms of leaving them a place in minimally decent conditions. That's why when you die for a cause, you're also dying so that tomorrow people can die less and suffer less. But in fact you're not dying for unborn people - you're dying for living ones from the future.
But to die to make room for others is simply to die for unborn people. Because them never being born is no tragedy - they never existed, so they never missed anything. But living people actually dying is a tragedy.
And I'm not against the fact that giving live is a great...
I can see the altruism in dying for a cause. But it's a leap of faith to claim, from there, that there's altruism in dying by itself. To die why, to make room for others to get born? Unborn beings don't exist, they are not moral patients. It would be perfectly fine if no one else was born from now on - in fact it would be better than even 1 single person dying.
Furthermore, if we're trying to create a technological mature society capable of discovering immortality, perhaps much sooner will it be capable of colonizing other planets. So there are trillions of...
To each paragraph:
Totally unfair comparison. Do you really think that immortality and utopia are frivolous goals? So maybe you don't really believe in cryonics or something. Well, I don't either. But transhumanism is way more than that. I think that its goals with AI and life extension are all but a joke.
That's reductive. As an altruist, I care about all other conscious being. Of course maintaining sanity demands some distancing, but that's that. So I'd say I'm a collectivist. But one person doesn't substitute the other. Others continuing to live wil
"You are a clone of your dead childhood self."
Yes, that's a typical Buddhist-like statement, that we die and are reborn each instant. But I think it's just incorrect - my childhood self never died. He's alive right now, here. When I die the biological death, then I will stop existing. It's as simple as that. Yet I feel like Buddhists, or Eastern religion in general, does this and other mental gymnastics to comfort people.
"So you either stick with modernism (that transhumanism is the one, special ideology immune from humanity's tragic need to self-sedate), ...
Still, that could all happen with philosophical zombies. A computer agent (AI) doesn't sleep and can function forever. These 2 factors is what leads me to believe that computers, as we currently define them, won't ever be alive, even if they ever come to emulate the world perfectly. At best they'll produce p-zombies.
"I'm feeling enthusiastic to try to make it work out, instead of being afraid that it won't."
Well, for someone who's accusing me of emotionally still defending a wrong mainstream norm (deathism), you're also doing it yourself by espousing empty positivism. Is it honest to feel enthusiastic about something when your probabilities are grim? The probabilities should come first, not how you feel about it.
"It's true that I lack the gear-level model explainig how it's possible for me to exist for quadrillion years."
Well I do have one to prove the opposite: the b...
It does ring true to me a bit. How could it not, when one cannot imagine a way to exist forever with sanity? Have you ever stopped to imagine, just relying on your intuition, what would be like to live for a quadrillion years? I'm not talking about a cute few thousand like most people imagine when we talk about immortality. I'm talking about proper gazillions, so to speak. Doesn't it scare the sh*t out of you? Just like Valentine says in his comment, it's curious how very few transhumanists have ever stopped to stare at this abyss.
On the other hand I don't...
"Whatever posttranshuman creature inherits the ghost of your body in a thousand years won't be "you" in any sense beyond the pettiest interpretation of ego as "continuous memory""
I used to buy into that Buddhist perspective, but I no longer do. I think that's a sedative, like all religions. Though I will admit that I still meditate, because I still hope to find out that I'm wrong. I hope I do, but I don't have a lot of hope. My reason and intuition are clear in telling me that the self is extremely valuable, both mine and that of all other conscious beings...
"if one might make a conscious being out of Silicon but not out of a Turing machine"
I also doubt that btw.
"what happens when you run the laws of physics on a Turing machine and have simulated humans arise"
Is physics computable? That's an open question.
And more importantly, there's no guarantee that the laws of physics would necessarily generate conscious beings.
Even if it did, could be p-zombies.
"What do you mean by "certainly exists"? One sure could subject someone to an illusion that he is not being subjected to an illusion."
True. But as long as you have...
Can we really separate them? I'm sure that the limitations of consciousness (software) have a physical base (hardware). I'm sure we could find the physical correlates of "failure to keep up with experience", as well as we could find the physical correlates of why someone who doesn't sleep for a few days starts failing to keep up with experience as well.
It all translates down to hardware at the end.
But anyway I'll say again that I admitted it was speculative and not the best example.
"There are now machine models that can recognize faces with mere compute, so probably the part of you that suggests that a cloud looks like a face is also on the outside."
Modern computers could theoretically do anything that a human does, except experience it. I can't draw a line around the part of my brain responsible for it because there is probably none, it's all of it. Even though I'm no neurologist. But from the little I know the brain has an integrated architecture.
Maybe in the future we could make conscious silicon machines (or of whatever material)...
I think it is factually correct that we get Alzheimer's and dementia at old age because the brain gets worn out. Whether it is because of failing to keep up with all the memory accumulation could be more speculative. So I admit that I shouldn't have made that claim.
But the brain gets worn out from what? Doing its job. And what's its job...?
Anyway, I think it would be more productive to at least present an explanation in a couple of lines rather than only saying that I'm wrong.
"So, odds are, you would not need to explicitly delete anything, it fades away with disuse."
I don't know. Even some old people feel overwhelmed with so many memories. The brain does some clean-up, for sure. But I doubt whether it would work for really long timelines.
"So, I don't expect memory accumulation to be an obstacle to eternal youth. Also, plenty of time to work on brain augmentations and memory offloading to external storage :)"
Mind you that your personal identity is dependent on your "story", which has to encompass all your life, even if only the ...
"No. There is nothing I find inherently scary or unpleasant about nonexistence."
Would you agree that you're perhaps a minority? That most people are scared/depressed about their own mortality?
"I'm just confused about the details of why that would happen. I mean, it would be sad if some future utopia didn't have a better solution for insanity or for having too many memories, than nonexistence.
Insanity: Look at the algorithm of my mind and see how it's malfunctioning? If nothing else works, revert my mindstate back a few months/years?
Memories: offload into l...
"Do I choose between being forced to exist forever, or to die after less than 100 years of existence? Neither. I'd like to have the option to keep living for as long as I want."
I didn't mean being forced to exist forever, or pre-commiting to anything. I meant that I really do WANT to exist forever, yet I can't see a way that it can work. That's the dilemma that I mentioned: to die, ever, even after a gazillion years, feels horrible, because YOU will cease to exist, no matter after how much time. To never die feels just as horrible because I can't see a way...
Thanks for the feedback (and the back-up). Well, I'd say that half of what I write on Lesswrong is downvoted and 40% is ignored, so I don't really care at this point. I don't think (most of) my opinions are outlandish. I never downvote anything I disagree with myself, and there's plenty that I disagree with. I only downvote absurd or rude comments. I think that's the way it should be, so I'm in full agreement on that.
You also got it right on my main point. That's precisely it. Mind you that "ending" consciousness also feels horrid to me! That's the dilemma. Would be great if we could find a way to achieve neverending consciousness without it being horrid.
" I can’t imagine getting bored with life even after a few centuries."
Ok, but that's not a lot of time, is it? Furthermore, this isn't even a question of time. For me, no finite time is enough. It's the mere fact of ceasing to exist. Isn't it simply horrifying? Even if you live a million healthy years, no matter, the fact is that you will cease to exist one day. And then there will be two options. Either your brain will be "healthy" and therefore will dread death as much as it does now, or it will be "unhealthy" and welcome death to relieve it's poor condi...
"I don't think we need to answer these questions to agree that many people would prefer to live longer than they currently are able."
Certainly, but that's not the issue here. The issue here is immortality. Many transhumanists desire to live forever, literally. Myself included. In fact I believe that many people in general do. Extending the human lifespan to thousands of years would be a great victory already, but that doesn't invalidate the search for true immortality, if people are interested in such which I'm sure some are.
"I have no idea what problems n...
1.) Suffering seems to need a lot of complexity, because it demands consciousness, which is the most complex thing that we know of.
2.) I personally suspect that the biological substrate is necessary (of course that I can't be sure.) For reasons, like I mentioned, like sleep and death. I can't imagine a computer that doesn't sleep and can operate for trillions of years as being conscious, at least in any way that resembles an animal. It may be superintelligent but not conscious. Again, just my suspicion.
3.) I think it's obvious - it means that we are trying...
Consciousness definitely serves a purpose, from an evolutionary perspective. It's definitely an adaptation to the environment, by offering a great advantage, a great leap, in information processing.
But from there to say that it is the only way to process information goes a long way. I mean, once again, just think of the pocket calculator. Is it conscious? I'm quite sure that it isn't.
I think that consciousness is a very biological thing. The thing that makes me doubt the most about consciousness in non-biological systems (let alone in the current ones whic...
I highly doubt this on an intuitive level. If a draw a picture of a man being shot, is it suffering? Naturally not, since those are just ink pigments in a sheet of cellulose. Suffering seems to need a lot of complexity and also seems deeply connected to biological systems. AI/computers are just a "picture" of these biological systems. A pocket calculator appears to do something similar to the brain but in reality it's much less complex and much different, and it's doing something completely different. In reality it's just an electric circuit. Are lightbulbs moral patients?
Now, we could someday crack consciousness in electronic systems, but I think it would be winning the lottery to get there not on purpose.
"That said, I'm not convinced that permanent Holocaust is worse than permanent extinction, but that's irrelevant to my point anyway."
Maybe it's not. What we guess are other people's values is heavily influenced by our own values. And if you are not convinced that permanent Holocaust is worse than permanent extinction, then, no offense, but you have a very scary value system.
"If someone isn't convinced by the risk of permanent extinction, are you likely to convince them by the (almost certainly smaller) risk of permanent Holocaust instead?"
Naturally, becaus...
"I love violence and would hope that Mars is an utter bloodbath."
The problem is that biological violence hurts like hell. Even most athletes live with chronic pain, imagine most warriors. Naturally we could solve the pain part, but then it wouldn't be the violence I'm referring to. It would be videogame violence, which I'm ok with since it doesn't cause pain or injury or death. But don't worry, I still got the joke!
""don't kill anyone and don't cause harm/suffering to anyone"
The problem with this one is that the AI's optimal move is to cease to exist."
I've...
"Plenty of 'failed utopia'-type outcomes that aren't exactly what we would ideally want would still be pretty great, but the chances of hitting them by accident are very low."
I'm assuming you've read Eliezer's post "Failed Utopia 4-2", since you use the expression? I've actually been thinking a lot about that, how that specific "failed utopia" wasn't really that bad. In fact it was even much better than the current world, as disease and aging and I'm assuming violence too got all solved at the cost of all families being separated for a few decades, which i...
"From the point of view of most humans, there are few outcomes worse than extinction of humanity (x-risk)."
That's obviously not true. What would you prefer: extinction of humanity, or permanent Holocaust?
"Are you implying that most leaders would prefer extinction of humanity to some other likely outcome, and could be persuaded if we focused on that instead?"
Anyone would prefer extinction to say a permanent Holocaust. Anyone sane at least. But I'm not implying that they would prefer extinction to a positive outcome.
"but I also think that those unpersuaded b...
"I mean, I agree that we've failed at our goal. But "haven't done a very good job" implies to me something like "it was possible to not fail", which, unclear?"
Of course it was. Was it difficult? Certainly. So difficult that I don't blame anyone for failing, like I've stated in my comment reply to this post.
It's an extremely difficult problem both technically and politically/socially. The difference is that I don't see any technical solutions, and have as well heard very convincing arguments by the likes of Roman Yalmpolskiy that such thing might not even e...
"I've come to the conclusion that it is impossible to make an accurate prediction about an event that's going to happen more than three years from the present, including predictions about humanity's end."
Correct. Eliezer has said this himself, check out his outstanding post "There is no fire alarm for AGI". However, you can still assign a probability distribution to it. Say, I'm 80% certain that dangerous/transformative AI (I dislike the term AGI) will happen in the next couple of decades. So the matter turns out to be just as urgent, even if you can't pre...
I'm of the opinion that we should tell both the politicians and the smart nerds. In my opinion we just haven't done a very good job. Maybe because we only focus on x-risk, and imo people in power and perhaps people in general might have internalized that we won't survive this century one way or another, be it climate change, nuclear, nano, bio or AI. So they don't really care. If we told them that unaligned AI could also create unbreakable dictatorships for their children perhaps it could be different. If people think of unaligned AI as either Heaven (Kurz...
In my opinion we just haven't done a very good job.
I mean, I agree that we've failed at our goal. But "haven't done a very good job" implies to me something like "it was possible to not fail", which, unclear?
We've seen plenty of people jump on the AI safety bandwagon.
Jumping on the bandwagon isn't the important thing. If anything, it's made things somewhat worse; consider the reaction to this post if MIRI were 'the only game in town' for alignment research as opposed to 'one lab out of half a dozen.' "Well, MIRI's given up," someone might say, "good thing ...
18 months is more than enough to get a DSA if AGI turns out anything we fear (that is, something really powerful and difficult to control, probably arriving fast at such state through an intelligence explosion).
In fact, I'd even argue 18 days might be enough. AI is already beginning to solve protein folding (Alphafold). If it progresses from there and builds a nanosystem, that's more than enough to get a DSA aka take over the world. We currently see AIs like MuZero learning in hours what would take a lifetime for a human to learn, so it wouldn't surprise m...
"what I really don't understand is why 'failure to solve the problem in time' sounds so much like 'we're all going to die, and that's so certain that some otherwise sensible people are tempted to just give in to despair and stop trying at all' "
I agree. In this community, most people only talk of x-risk (existential risk). Most people equate failure to align AI to our values to human extinction. I disagree. Classic literature examples of failure can be found, like With Folded Hands, where AI creates an unbreakable dictatorship, not extinction.
I think it's ...
A loud strategy is definitely mandatory. Just not too loud with the masses. Only loud with the politicians, tech leaders and researchers. We must convince them that this is dangerous. More dangerous than x-risk even. I know that power corrupts, but I don't think any minimally sane human wants to destroy everything or worse. The problem imo is that they aren't quite convinced, nor have we created strong cooperative means in this regard.
So far this is kinda being done. People like Stuart Russell are quite vocal. Superintelligence by Nick Bostrom had a huge impact. But despite all these great efforts it's still not enough.
A loud strategy is definitely mandatory. Just not too loud with the masses. Only loud with the politicians, tech leaders and researchers.
One way to characterize the thing that happened over the last ten years is: "oh no, we have a two-part argument: 1) AI very powerful and 2) that could be very dangerous. If we tell politicians about that, they might only hear the first part and make it worse. Let's tell smart nerds instead." Then, as Eli points out, as far as we can tell on net the smart nerds heard the first part and not the second.
M. Y. Zuo, what you describe is completely possible. The problem is that such positive outcome, as well as all others, is extremely uncertain! It's a huge gamble. Like I've said in other posts: would you spin a wheel of fortune with, let's say, 50% probability of Heaven, and 50% probability of extinction or worse?
Let me tell you that I wouldn't spin that wheel, not even with a 5% probability of bad outcomes only. Alignment is about making sure that we reduce that probability to a low as we can. The stakes are super high.
And if like me you place a low probability of acceptable outcomes without alignment solved, then it becomes even more imperative.
Could be. I'll concede that the probability that the average person couldn't effectively do anything is much higher than the opposite. But imo some of the probable outcomes are so nefarious that doing nothing is just not an option, regardless. After all, if plenty of average people effectively decided to do something, something could get done. A bit like voting - one vote achieves nothing, many can achieve something.
I'm sorry but doing nothing seems unnaceptable to me. There are some in this forum who have some influence on AI companies, so those could definitely do something. As for the public in general, I believe that if a good number of people took AI safety seriously, so that we could make our politicians take it seriously, things would change.
So there would definitely be the need to do something. Specially because unfortunately this is not a friendly AI / paperclipper dicotomy like most people here present it by only considering x-risk and not worse outcomes. I can imagine someone accepting death because we've always had to accept it, but not something worse than it.
I don't buy that even Buddha himself could avoid the suffering from the most severe forms of pain, which are also the ones that matter most - which makes it pretty arrogant to claim that suffering is optional imo.