= 27d567bc2d48d9f40e9ccc20ea370b15)
This is a lot more interesting a response if you would also agree with Lalartu in the more general case.
Kind of. I wouldn't defect against my copy without his consent, but I would want the pool trimmed down to only a single version of myself (ideally whichever one had the highest expected future utility, all else equal). The copy, being a copy, should want the same thing. The only time I wouldn't be opposed to the existence of multiple instances of myself would be if those instances could regularly synchronize their memories and experiences (and thus constitute more a single distributed entity with mere synchronization delays than multiple diverging entities).
I keep returning to one gnawing question that haunts the whole idea of Friendly AI: how do you program a machine to "care"? I can understand how a machine can appear to "want" something, favoring a certain outcome over another. But to talk about a machine "caring" is ignoring a very crucial point about life: that as clever as intelligence is, it cannot create care. We tend to love our kid more than someone else's. So you could program a machine to prefer another in which it recognizes a piece of its own code. That may LOOK like care but it's really just an outcome. How could you replicate, for example, the love a parent shows for a kid they didn't produce? What if that kid were humanity? So too with Coherent Extrapolated Volition, you can keep refining the resolution of an outcome, but I don't see how any mechanism can actually care about anything but an outcome.
While "want" and "prefer"may be useful terms, such terms as "care", "desire", "value" constitute an enormous and dangerous anthropomorphizing. We cannot imagine outside our own frame, and this is one place where that gets us into real trouble. Once someone creates a code that will recognize something truly metaphysical I would be convinced that FAI is possible. Even whole brain emulation assumes that both that our thoughts are nothing but code and a brain with or without a body is the same thing. Am I missing something?
Leaving aside other matters, what does it matter if an FAI 'cares' in the sense that humans do so long as its actions bring about high utility from a human perspective?
What would your copy want?
What if it was a near-copy without $fatalMedicalCondition?
My first thought (in response to the second question) is 'immediately terminate myself, leaving the copy as the only valid continuation of my identity'.
Of course, it is questionable whether I would have the willpower to go through with it. I believe that my copy's mind would constitute just as 'real' a continuation of my consciousness as would my own mind following a procedure that removed the memories of the past few days (or however long since the split) whilst leaving all else intact (which is of course just a contrived-for-the-sake-of-the-thought-experiment variety of the sort of forgetting that we undergo all the time), but I have trouble alieving it.
In response to
Stuart Russell: AI value alignment problem must be an "intrinsic part" of the field's mainstream agenda
I'm a super-dummy when it comes to thinking about AI. I rightly leave it to people better equipped and more motivated than me.
But, can someone explain to me why a solution would not involve some form of "don't do things to people or their property without their permission"? Certainly, that would lead to a sub-optimal use of AI in some people's opinions. But it would completely respect the opinions of those who disagree.
Recognizing that I am probably the least AI-knowledgeable person to have posted a comment here, I ask, what am I missing?
In response to
comment
by
on
Stuart Russell: AI value alignment problem must be an "intrinsic part" of the field's mainstream agenda
Even leaving aside the matters of 'permission' (which lead into awkward matters of informed consent) as well as the difficulties of defining concepts like 'people' and 'property', define 'do things to X'. Every action affects others. If you so much as speak a word, you're causing others to undergo the experience of hearing that word spoken. For an AGI, even thinking draws a miniscule amount of electricity from the power grid, which has near-negligible but quantifiable effects on the power industry which will affect humans in any number of different ways. If you take chaos theory seriously, you could take this even further. It may seem obvious to a human that there's a vast difference between innocuous actions like those in the above examples and those that are potentially harmful, but lots of things are intuitively obvious to humans and yet turn out to be extremely difficult to precisely quantify, and this seems like just such a case.
Terminal values are what are sought for their own sake, as opposed to instrumental values, which are sought because they ultimately produce terminal values.
I know what terminal values are and I apologize if the intent behind my question was unclear. To clarify, my request was specifically for a definition in the context of human beings - that is, entities with cognitive architectures with no explicitly defined utility functions and with multiple interacting subsystems which may value different things (ie. emotional vs deliberative systems). I'm well aware of the huge impact my emotional subsystem has on my decision making. However, I don't consider it 'me' - rather, I consider it an external black box which interacts very closely with that which I do identify as me (mostly my deliberative system). I can acknowledge the strong influence it has on my motivations whilst explicitly holding a desire that this not be so, a desire which would in certain contexts lead me to knowingly make decisions that would irreversibly sacrifice a significant portion of my expected future pleasure.
To follow up on my initial question, it had been intended to lay the groundwork for this followup: What empirical claims do you consider yourself to be making about the jumble of interacting systems that is the human cognitive architecture when you say that the sole 'actual' terminal value of a human is pleasure?
My position is in line with that - people are wrong about what their terminal values are, and they should realize that their actual terminal value is pleasure.
Can you define 'terminal values', in the context of human beings?
all we can do is a finite amount of good or evil (which has no impact on an infinite value)
If the universe is infinite, then there are infinitely many copies of me, following the same algorithm, so my decisions create infinite amounts of good or evil (through my copies which decide the same way).
Or, to see it from another angle, if the universe is literally infinite, then it is more or less infinitely repetitive. So let's take a part of universe containing a copy of approximately everything, and treat this part as a finite universe, which is just replicated infinitely many times.
Your "measure" is the proportion of your copies to the infinite universe.
If the universe is infinite, then there are infinitely many copies of me, following the same algorithm
Does this follow? The set of computable functions is infinite, but has no duplicate elements.
In response to
Fighting Biases and Bad Habits like Boggarts
"Comments (1)"
"There doesn't seem to be anything here."
????
= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
Because what any human wants is a moving target. As soon as someone else delivers exactly what you ask for, you will be disappointed unless you suddenly stop changing. Think of the dilemma of eating something you know you shouldn't. Whatever you decide, as soon as anyone (AI or human) takes away your freedom to change your mind, you will likely rebel furiously. Human freedom is a huge value that any FAI of any description will be unable to deliver until we are no longer free agents.
What would an AI that 'cares' in the sense you spoke of be able to do to address this problem that a non-'caring' one wouldn't?