Comment author: snarles 25 July 2015 04:05:59AM *  5 points [-]

There is no way to raise a human safely if that human has the power to exponentially increase their own capabilities and survive independently of society.

Comment author: kokotajlod 26 July 2015 08:35:56PM 0 points [-]

Yep. "The melancholy of haruhi suzumiya" can be thought of as an example of something in the same reference class.

In response to Counterfactual trade
Comment author: kokotajlod 05 June 2015 07:56:53PM 0 points [-]

This is an interesting idea! Some thoughts:

Doesn't acausal trade (like all trade) depend on enforcement mechanisms? I can see how two AI's might engage in counterfactual trade, since they can simulate each other and see that they self-modify to uphold the agreement, but I don't think a human would be able to do it.

Also, I'd like to hear more about motivations for engaging in counterfactual trade. I get the multiverse one, though I think that's a straightforward case of acausal trade rather than a case of counterfactual trade, since you would be trading with a really existing entity in another universe. But can you explain the second motivation more?

Comment author: lackofcheese 29 September 2014 01:22:34AM 1 point [-]

It seems to me that you should only do this if everyone has utility functions that are completely anthropically selfish (i.e. they only care about their own subjective experience). Otherwise, wouldn't it be cruel to intentionally simulate a world with so many unpleasant characteristics that we could otherwise remove if we weren't focused on making the simulation subjectively indistinguishable from our own world?

As such, I don't think we should commit to any such thing.

Comment author: kokotajlod 30 September 2014 10:48:03AM 1 point [-]

The point you raise is by far the strongest argument I know of against the idea.

However, it is a moral objection rather than a decision-theory objection. It sounds like you agree with me on the decision theory component of the idea: that if we were anthropically selfish, it would be rational for us to commit to making ancestor-simulations with afterlives. That's an interesting result in itself, isn't it? Let's go tell Ayn Rand.

When it comes to the morality of the idea, I might end up agreeing with you. We'll see. I think there are several minor considerations in favor of the proposal, and then this one massive consideration against it. Perhaps I'll make a post on it soon.

Comment author: kokotajlod 28 September 2014 10:34:03PM 6 points [-]

This is a formal version of a real-life problem I've been thinking about lately.

Should we commit to creating ancestor-simulations in the future, where those ancestor-simulations will be granted a pleasant afterlife upon what appears to their neighbors to be death? If we do, then arguably we increase the likelihood that we ourselves have a pleasant afterlife to look forward to.

Comment author: helltank 13 September 2014 11:46:08PM 2 points [-]

The thing is, if you get suspicious you don't immediately leap to the conclusion of chatbots. Nobody glances around, realizes everyone is bland and stupid and thinks," I've been fooled! An AI has taken over the world and simulated chatbots as human beings!" unless they suffer from paranoia.

Your question, "How much more sophisticated would they need to be" is answered by the question "depends". If you live as a hermit in a cave up in the Himalayas, living off water from a mountain stream and eating nothing but what you hunt or gather with your bare hands, the AI will not need to use chatbots at all. If you're a social butterfly who regularly talks with some of the smartest people in the world, the AI will probably struggle(let's be frank; if we were introduced to a chatbot imitating Eliezer Yudkosky the difference would be fairly obvious).

If you've interacted a lot with your friends and family, and have never been once suspicious that they are a chatbot, then with our current level of AI technology it is unlikely(but not impossible) that they are actually chatbots.

[Please note that if everyone says what you expect them to say that would make it fairly obvious something is up, unless you happen to be a very, very good predictor of human behavior.]

Comment author: kokotajlod 14 September 2014 03:47:59PM *  1 point [-]

Thanks for the response. Yes, it depends on how much interaction I have with human beings and on the kind of people I interact with. I'm mostly interested in my own case, of course, and I interact with a fair number of fairly diverse, fairly intelligent human beings on a regular basis.

If you're a social butterfly who regularly talks with some of the smartest people in the world, the AI will probably struggle

Ah, but would it? I'm not so sure, that's why I made this post.

Yes, if everyone always said what I predicted, things would be obvious, but recall I specified that random variation would be added. This appears to be how dream characters work: You can carry on sophisticated conversations with them, but (probably) they are governed by algorithms that feed off your own expectations. That being said, I now realize that the variation would have to be better than random in order to account for how e.g. EY consistently says things that are on-point and insightful despite being surprising to me.

Comment author: Izeinwinter 13 September 2014 10:29:43AM 0 points [-]

Consider the Tatiana Maslany as Mara the deceiver strategy. There is already one true person in the system other than you. The AI. That means everyone you interact with can be arbitrarily sophisticated without adding any more people to the system - All it takes is for the AI to directly puppet them. Acting in place of making bots.

This is, however, not a fruitful line of thinking, If the universe were really out to get you to this degree you would be better of being oblivious.

Comment author: kokotajlod 13 September 2014 12:40:00PM 1 point [-]

Yes, but I'm not sure there is a difference between an AI directly puppeting them, and an AI designing a chatbot to run as a subroutine to puppet them, at least if the AI is willing to monitor the chatbot and change it as necessary. Do you think there is?

Also, it totally is a fruitful line of thinking. It is better to believe the awful truth than a horrible lie. At least according to my values. Besides, we haven't yet established that the truth would be awful in this case.

Comment author: HungryHobo 12 September 2014 11:09:12AM 2 points [-]

Define "make them people". Because it sounds like what you're talking about is just the problem of other minds with "chatbot" substituted for "zombie"

Comment author: kokotajlod 12 September 2014 11:43:44PM 0 points [-]

I'm surprised that it sounded that way to you. I've amended my original post to clarify.

Comment author: [deleted] 12 September 2014 01:02:55PM 2 points [-]

The amount of detail an AI would need for simulating realistic NPC's for you may be influenced substantially by a significant list of things like whether you are an introvert or an extrovert, what your job is, how many people you interact with over the course of a day and to what level of detail, and how many of those people you have very deep conversations with and how frequently, and if an acceptable AI answer to you mentioning to someone 'Everyone and everything seems so depressingly bland and repetitive' is a doctor telling you 'Have you tried taking medication? I'll write you a prescription for an antidepressant.'

Comment author: kokotajlod 12 September 2014 11:24:17PM 1 point [-]

Yes, this is the sort of consideration I had in mind. I'm glad the discussion is heading in this direction. Do you think the answer to my question hinges on those details though? I doubt it.

Perhaps if I was extraordinarily unsuspicious, chatbots of not much more sophistication than modern-day ones could convince me. But I think it is pretty clear that we will need more sophisticated chatbots to convince most people.

My question is, how much more sophisticated would they need to be? Specifically, would they need to be so much more sophisticated that they would be conscious on a comparable level to me, and/or would require comparable processing power to just simulating another person? For example, I've interacted a ton with my friends and family, and built up detailed mental models of their minds. Could they be chatbots/npcs, with minds that are nothing like the models I've made?

(Another idea: What if they are exactly like the models I've made? What if the chatbot works by detecting what I expect someone to say, and then having them say that, with a bit of random variation thrown in?)

Comment author: HungryHobo 11 September 2014 11:48:10PM 5 points [-]

How long is a piece of string?

If you did live in such a world where everything was based around you then the controller could allocate resources even more efficiently by monitoring your mind and devoting more computational power to making the people around you believable when you get suspicious.

Comment author: kokotajlod 12 September 2014 10:45:29AM 1 point [-]

That's exactly what I had in mind, although I did specify that the controller would never simulate anybody besides me to the level required to make them people.

How realistic would AI-engineered chatbots be?

-1 kokotajlod 11 September 2014 11:00PM

I'm interested in how easy it would be to simulate just one present-day person's life rather than an entire planet's worth of people. Currently our chatbots are bad enough that we could not populate the world with NPC's; the lone human would quickly figure out that everyone else was... different, duller, incomprehensibly stupid, etc.

But what if the chatbots were designed by a superintelligent AI?

If a superintelligent AI was simulating my entire life from birth, would it be able to do it (for reasonably low computational resources cost, i.e. less than the cost of simulating another person) without simulating any other people in sufficient detail that they would be people?

I suspect that the answer is yes. If the answer is "maybe" or "no," I would very much like to hear tips on how to tell whether someone is an ideal chatbot.

Thoughts?

EDIT: In the comments most people are asking me to clarify what I mean by various things. By popular demand:

I interact with people in more ways than just textual communication. I also hear them, and see them move about. So when I speak of chatbots I don't mean bots that can do nothing but chat. I mean an algorithm governing the behavior of a simulated entire-human-body, that is nowhere near the complexity of a brain. (Modern chatbots are algorithms governing the behavior of a simulated human-hands-typing-on-keyboard, that are nowhere near the complexity of a brain.)

When I spoke of "simulating any other people in sufficient detail that they would be people" I didn't mean to launch us into a philosophical discussion of consciousness or personhood. I take it to be common ground among all of us here that very simple algorithms, such as modern chatbots, are not people. By contrast, many of us think that a simulated human brain would be a person. Assuming a simulated human brain would be a person, but a simple chatbot-like algorithm would not, my question is: Would any algorithm complex enough to fool me into thinking it was a person over the course of repeated interactions actually be a person? Or could all the bodies around me be governed by algorithms which are too simple to be people?

I realize that we have no consensus on how complex an algorithm needs to be to be a person. That's OK. I'm hoping that this conversation can answer my questions anyhow; I'm expecting answers along the lines of

(A) "For a program only a few orders of magnitude more complicated than current chatbots, you could be reliably fooled your whole life" or

(B) "Any program capable of fooling you would either draw from massive databases of pre-planned responses, which would be impractical, or actually simulate human-like reasoning."

These answers wouldn't settle the question for good without a theory of personhood, but that's OK with me, these answers would be plenty good enough.

Comments (17)

View more: Prev | Next

Powered by Reddit Powered by Reddit