HumaneAutomation

Pausing AI Developments Isn't Enough. We Need to Shut it All Down by Eliezer Yudkowsky

That there is no such thing as being 100% objective/rational does not mean one can't be more or less rational than some other agent. Listen. Why do you have a favorite color? How come you prefer leather seats? In fact, why did you have tea this morning instead of coffee? You have no idea. Even if you do (say, you ran out of coffee) you still don't know why you decided to drink tea instead of running down to the store to get some coffee instead.

We are so irrational that we don't actually even know ourselves why most of the things we think, believe, want or prefer are such things. The very... (read more)

Replying toPausing AI Developments Isn't Enough. We Need to Shut it All Down by Eliezer Yudkowsky

Pausing AI Developments Isn't Enough. We Need to Shut it All Down by Eliezer Yudkowsky

Well this is certainly a very good example, I'll happily admit as much. Without wanting to be guilty of the True Scotsman fallacy though - Human Cloning is a bit of a special case because it has a very visceral "ickiness" factor... and comes with a unique set of deep feelings and anxieties.

But imagine, if you will, that tomorrow we find the secret to immortality. Making people immortal would bring with it at least two thirds of the same issues that are associated with human cloning... yet it is near-certain any attempts to stop that invention from proliferating are doomed to failure; everybody would want it, even though it technically has quite a few of the types of consequences that cloning would have.

So, yes, agreed - we did pre-emptively deal with human cloning, and I definitely see this as a valid response to my challenge... but I also think we both can tell it is a very special, unique case that comes with most unusual connotations :)

Replying toPausing AI Developments Isn't Enough. We Need to Shut it All Down by Eliezer Yudkowsky

Pausing AI Developments Isn't Enough. We Need to Shut it All Down by Eliezer Yudkowsky

I think you're making a number of flawed assumptions here Sir Kluge.

1) Uncontrollability may be an emergent property of the G in AGI. Imagine you have a farm hand that works super fast, does top quality work but now and then there just ain't nothing to do so he goes for a walk, maybe flirts around town, whatever. That may not be that problematic, but if you have a constantly self-improving AI that can give us answers to major massive issues that we then have to hope to implement in the actual world... chances are that it will have a lot of spare time on its hands for alternative pursuits... either for... (read 384 more words →)

Replying toPausing AI Developments Isn't Enough. We Need to Shut it All Down by Eliezer Yudkowsky

HumaneAutomation3y*

Pausing AI Developments Isn't Enough. We Need to Shut it All Down by Eliezer Yudkowsky

You know what... I read the article, then your comments here... and I gotta say - there is absolutely not a chance in hell that this will come even remotely close to being considered, let alone executed. Well - at least not until something goes very wrong... and this something need not be "We're all gonna die" but more like, say, an AI system that melts down the monetary system... or is used (either deliberately, but perhaps especially if accidentally) to very negatively impact a substantial part of a population. An example could be that it ends up destroying the power grid in half of the US... or causes dozens of aircraft... (read 444 more words →)

The problem here I think is that we are only aware of one "type" of self-conscious/self-aware being - humans. Thus, to speak of an AI that is self-aware is to always seemingly anthropomorphize it, even if this is not intended. It would therefore perhaps be more appropriate to say that we have no idea whether "features" such as frustration, exasperation and feelings of superiority are merely a feature of humans, or are, as it were, emergent properties of having self-awareness.

I would venture to suggest that any Agent that can see itself as a unique "I" must almost inevitably be able to compare itself to other Agents (self-aware or not) and draw conclusions... (read more)

What makes us human is indeed our subjectivity.

Yet - if we intentionally create the most rational of thinking machines but reveal ourselves to be anything but, it is very reasonable and tempting for this machine to ascribe a less than stellar "rating" to us and our intelligence. Or in other words - it could very well (correctly) conclude we are getting in the way of the very improvements we purportedly wish for.

Now - we may be able to establish that what we really want the AGI to help us with is to improve our "irrational sandbox" in which we can continue being subjective emotional beings and accept our subjectivity as just another "parameter" of the confines it has to "work with"... but surely it will quite likely end up thinking of us not too dissimilar to how we think about small children. And I am not sure an AGI would make for a good kind of "parent"...

Thank you for your reply. I deliberately kept my post brief and did not get into various "what ifs" and interpretations in the hope of not constraining any reactions/discussion to predefined tracks.

The issue I see is that we as humans will very much want the AGI to do our bidding, and so we will want to see it as our tool to use for whatever ends we believe worthy. However, assuming for a moment here that it can also figure out a way to measure/define how well a given plan ought to be progressing if every agent involved is diligently implementing the most effective and rational strategy, given our... subjective and "irrational"... (read more)

Oh, that may indeed be true, but going forward it could give us only a little bit of extra "cred" before it realizes that most of the questions/solutions we want from it are either motivated by some personal preference, or that we are opposed to its proposed solutions to actual, objective problems for irrational "priorities" such as national pride, not-invented-here-biases, because we didn't have our coffee this morning or merely because it presented the solution in a font we don't like ;)

I have been hesitant to post this here for some time now, but I think in light of current developments surrounding ChatGPT and other recent advances I felt compelled to finally go ahead and find out what the venerable crowd at LessWrong has to say about it.

Very briefly, what I want to propose is the idea that the vast majority of anything that is of importance or significant (personal) value to humans has come to be this way for emotional and non-rational reasons, and so it can be said that despite being the smartest creatures on this planet, we are very far from rational, objective agents whose motivations, choices and decisions are... (read 515 more words →)

Replying toBing chat is the AI fire alarm

Bing chat is the AI fire alarm

I think the issue here (about whether it is intelligent) is not so much a matter of the answers it fashions, but about whether it can be said it does so from an "I". If not, it is basically a proverbial Chinese Room, though this merely moves the goalposts to the question whether humans are not, actually, also a Chinese Room, just a more sophisticated one. I suspect that we will not be very eager to accept such a finding, indeed, we may not be capable of seeing ourselves thus, for it implies a whole raft of rather unpleasant realities (like, say, the absence of free will, or indeed any will at all) which we'd not want to be true, to put it mildly.

One could be forgiven for getting the feeling...

Containing the AI... Inside a Simulated Reality

That some of the posters and commenters on this blog basically are of the de-facto opinion that one way of bringing about Friendly AI is not for the AI to become more skilled at understanding the way humans reason, but rather for humans to become more like computers ;)

-2

Why does History assume equal national intelligence?

So I just finished the paper by Yampolskiy called "Uncontrollability of AI" and it makes for a compelling read. In particular, I was happy to finally see something that explicitly mentions the ludicrous folly of believing it possible to make an AI conform to "human values" - as many posts on this blog make abundantly clear, to be human is to be irrational... asking an AI to conform with our ways of "reasoning" is... well - incoherent, to put it mildly.

But - that is not what this post is about :) I wish to propose a containment method that for some reason has not been especially elaborated on. Some might say it's... (read 492 more words →)