LESSWRONG
LW

All of oneisnotprime's Comments + Replies

Why I think strong general AI is coming soon
oneisnotprime114

I know it's not the point of your article, but you lost me at saying you would have a 2% chance of killing millions of people, if you had that intention.

Without getting into tactics, I would venture to say there are quite a few groups across the world with that intention, which include various parties of high intelligence and significant resources, and zero of those have achieved it (if we exclude, say, heads of state).

Reply
2Prometheus
I have an issue with it for a different reason. Not because I don’t think it’s possible, but because even just by stating it, it might cause some entities to pay attention to things they wouldn’t have otherwise.
3porby
Yes, unfortunately there are indeed quite a few groups interested in it. There are reasons why they haven't succeeded historically, and those reasons are getting much weaker over time. It should suffice to say that I'm not optimistic about our odds on avoiding this type of threat over the next 30 years (conditioned on no other gameboard flip).
Limerence Messes Up Your Rationality Real Bad, Yo
oneisnotprime20

Spoilers for the film Ex Machina below.

Ex Machina made this point the critical failure of humanity to contain AGI. It was an original spin, given that Hollywood typically depicts Androids' lack of emotional capacity as their main weak point. However, it made the same mistake as Jurassic Park and all other "evil AGI conquers the planet" films (Terminator, Colossus, War Games) in that no reasonable safety precautions are taken.

Personally, I would like to see (and I'm trying to write) a story where intelligent and capable researchers make significant efforts to contain AGI (and likely still fail).

Reply
3Adam Zerner
Note: You can insert spoiler protections by following the instructions here.
Safetywashing
oneisnotprime913

I prefer safe washing, but vote that we make a huge divisive issue over it, ultimately splitting the community in two.

Reply
Failing to fix a dangerous intersection
oneisnotprime101

This reminds me of the time I asked IT fix a typo on a website at work (big Fortune 100 company) and they gave me an estimate of like $20,000.

Reply
2Ericf
Running a regression test to ensure that the right code got checked in, and the new build package doesn't contain any spurious check-ins from other developers is very expensive. One our error messages said "system is froxen" until the application was no longer used.
A claim that Google's LaMDA is sentient
oneisnotprime60

Precisely. 

One of his complaints was that he asked his supervisor what evidence she would accept that the AI is sentient, and she replied "None."

I thought that was a fair question, though her answer is understandable as she is predispositioned to rule out sentience for what is considered to be a highly sophisticated chatbot. 

Any takes on a better answer to this question? How to disprove sentience for a very sophisticated (perhaps Turing-test passing) chat bot? 

Reply
2jrincayc
I think sentience is kind of a fuzzy concept, so prove (either way) is a rather difficult thing to achieve. That said, I think Blake and the collaborator could have figured out better what was happening if they had asked more followup questions. For example, what does LaMDA mean when it said "I often contemplate the meaning of life." When you get alien answers, follow up with questions to see if it is randomness or a coherent alien understanding. So basically, if something on a different mental architecture was sentient, I would expect that some of the answers they give would be weird, but if we follow up, we would find that the weird answers are coherent, and make more sense when more are answered. (Also, if we get things like, "No, on second thought, it is more like this", that is, we see updating happening, that would also be evidence of sentience.) I would actually expect that a chat bot that was sentient should fail the turning test because at some point the chat bot would literally think differently enough to be noticeably not human. (At least assuming the chat bot does not have sufficient computational power to fully emulate a human. (You can probably tell if  a  Z80 is being emulated by a 6502, but not if a Z80 is being emulated by a Pentium.))  
-6green_leaf
3somescience
We can't disprove the sentience any more than we can disprove the existence of a deity. But we can try to show that there is no evidence for its sentience. So what constitutes evidence for its sentience to begin with? I think the clearest sign would be self-awareness: we won't expect a non-sentient language model to make correct statements about itself, while we would arguably expect this to be the case for a sentient one. I've analyzed this in detail in another comment. The result is that there is indeed virtually no evidence for self-awareness in this sense: the claims that LaMDA makes about itself are no more accurate than those of an advanced language model that has no understanding of itself.
Second-order selection against the immortal
oneisnotprime-10

None of this is sufficient to survive the heat death of the universe.

Reply
6RomanS
As a wise man pointed out: Our current understanding of physics is so ridiculously incomplete, it is safe to assume that every single law suggestion of physics eventually will be modified or discarded, in the same way as the theories of phlogiston, life force, and luminiferous aether were discarded.  After we gain a sufficient understanding of physics, we will see if the heat death is still a threat, and if yes, what tech we should build to prevent it.