Comment author: stcredzero 14 June 2012 07:18:28PM 2 points [-]

Come to think of it a suicidal AI could be a pretty big problem...

It's probably been thought of here and other places before, but I just thought of the "Whoops AI" -- a superhuman AGI that accidentally or purposefully destroys the human race, but then changes its mind and brings us back as a simulation.

Comment author: Logos01 14 June 2012 10:48:15PM -2 points [-]

but then changes its mind and brings us back as a simulation."

This is commonly referred to as a "counterfactual" AGI.

In response to comment by Logos01 on Be Happier
Comment author: tog 13 June 2012 01:44:17PM *  0 points [-]

Surely a self-proving value is one where the question "Is X valuable?" is self-proving?

In response to comment by tog on Be Happier
Comment author: Logos01 13 June 2012 09:41:32PM 0 points [-]

Indeed. Which is why happiness is not a terminal value.

Comment author: Eugine_Nier 02 June 2012 01:33:15AM 1 point [-]

over time your genetic pattern changes -- the composition of cancerous to non-cancerous cells; the composition of senescent to non-senescent cells

Note that neither of these developments are generally considered good.

Comment author: Logos01 02 June 2012 02:31:10AM 0 points [-]

Indeed. But they do demonstrate the principle in question.

Comment author: loup-vaillant 30 May 2012 01:20:44PM 1 point [-]

No, it doesn't. You could argue that there's a renewal of atoms (most notably water), but swapping water atoms doesn't have physical meaning, so… No. Heck, even cut&paste transportation doesn't change substrate.

The "yuck factor" I feel cause me to doubt this: If an EM of me would be created during my sleep, what probability would I assign to wake up as silicon, or as wetware? I'm totally not sure I can say 1/2.

Comment author: Logos01 01 June 2012 05:42:30PM 0 points [-]

Actually it's more complicated than that. Not just water atoms; over time your genetic pattern changes -- the composition of cancerous to non-cancerous cells; the composition of senescent to non-senescent cells; the physical structures of the brain itself change.

Neurogenesis does occur in adults -- so not even on a cellular level is your brain the same today as it was yesterday.

Furthermore -- what makes you confident you are not already in a Matrix? I have no such belief, myself. Too implausible to believe we are in the parent of all universes given physics simulations work.

Comment author: TheOtherDave 29 May 2012 06:29:10PM 0 points [-]

I agree that my meat does not persist over time.

The class of patterns of information-flow that can occur within meat includes the pattern of information-flow that occurs within your meat. 3 therefore asserts that I am you, in addition to being me. 2 does not assert this. They seem like different claims to me, insofar as any of these claims are different from the others.

I'm not really sure what non-local phenomena are, or what they have to do with psychic powers, or what they have to do with the proper referent for "I".

Comment author: Logos01 01 June 2012 05:38:53PM 0 points [-]

Missed that about the class. Makes a difference, definitely.

I'm not really sure what non-local phenomena are [...]

Two options: trust the assertions of those who are sure, or learn of them for yourself. :)

Comment author: TheOtherDave 23 May 2012 11:27:43PM 4 points [-]

Consider the following set of statements:
1) "I am my meat."
2) "I am the unique pattern of information-flow that occurs within my meat."
3) "I am the class of patterns of information-flow that can occur within meat, of which this unique pattern is one example."
4) "I am the class of patterns of information-flow that can occur within any substrate, of which this unique pattern is one example."
5) "I am all the matter and energy in the universe."

What sorts of experiences would constitute evidence for one of them over the others?

Comment author: Logos01 29 May 2012 05:32:52PM 0 points [-]

1 v 2 -- is your "meat" persistent over time? (It is not).

2 v 3 are non differentiable -- 2 is 3.

4 is implied by 2/3. It is affirmed by physics simulations that have atomic-level precision, and by research like the Blue Brain project.

5 is excluded by the absence of non-local phenomena ('psychic powers').

Comment author: loup-vaillant 24 May 2012 09:33:16PM 1 point [-]

I'm not sure that's a meaningful question. I undoubtedly change from year to year, so… But there is some kind of continuity, which I'm afraid could be broken by a change of substrate. (But then again, we could change my substrate bit by bit…

If it weren't, I would not care, because it wouldn't break anything I value. If preservation of identity doesn't even happen currently in our mundane world, I would be stupid to value it. And I'll happily upload, then (modulo the mundane risk of being badly emulated of course).

But first, I must be convinced that either identity wasn't preserved in the first place, or that uploading preserves identity, or that I was just confused because the world actually works like… who knows.

Comment author: Logos01 29 May 2012 05:29:25PM 0 points [-]

A change of substrate occurs daily for you. It's just of a similar class. What beyond simple "yuck factor" gives you cause to believe that a transition from cells to silicon would impact your identity? That it would look different?

Comment author: TimS 24 May 2012 01:14:53AM *  1 point [-]

True, but scientific truths that imply moral truths are very, very thin on the ground. (I personally doubt there are any scientific truths dispositive of most actually contested moral issues).

Comment author: Logos01 29 May 2012 05:27:00PM 0 points [-]

Scientific truths include the measurement of net harm to society for any given action -- which then impact utilitarian consequentialistic morals. ("It's unjust to execute anyone. Ever.")

Scientific truths include observations as to what occurs "in nature" which then informs naturalistic morals ("It's not natural to be gay/left-handed/brilliant" )

Scientific truths include observations about the role morality plays in those species we can observe to possess it, thereby informing us practically about what actions or inactions or rules would best optimize that function. (Observing apes and other primates or pack animals to derive a functional analysis of how morality impacts our social coherence and so on.)

I have long argued that morality needn't be absolute in order to be objective. Moral relativism and moral objectivism may be standard terms but I assert they are not as incompatible as is routinely claimed.

We needn't know what is perfectly moral to know objectively what is less moral.

Comment author: loup-vaillant 23 May 2012 02:50:47PM *  0 points [-]

Yes, I meant protoplasm.

If I knew that I am currently running on a silicon chip (Gunm-style), then I would be highly confident that replacing that chip by another, identical one, preserves my identity, because it's the same configuration. Moreover, replacing my old chip by a newer one, before the physical deterioration significantly affects the actual software processing, probably would work as well.

But if we're talking about running my software on a different chip through, say, a virtual machine that emulate my original chip, then I would be less confident that it would still be me. As confident as I am that, an EM of my current wetware would still be me. Which is, currently, not confident enough to make the leap.

Ah, and if I do learn that I run on a chip, I won't turn crazy. I may be worried if I knew my wetware self were still running around, and I may not tell my mother, but besides that I don't really care. If I knew that my wetware self was "dead", then I would wonder if I should feel sorry for him, or if I'm actually him. Because I value my life, I know that my wetware self did too. But I'd probably get over it with the knowledge that the rest of the world (including my family) didn't lose anything, (or at least they wouldn't suspect a thing).

I'm confident an EM would not be a PZombie.

Comment author: Logos01 23 May 2012 09:59:01PM 0 points [-]

As I often say; you are not your meat. You are the unique pattern of information-flow that occurs within your meat. The meat is not necessary to the information, but the information does require a substrate.

Comment author: EHeller 23 May 2012 03:28:20AM 3 points [-]

I think you and your mother might be conflating different meanings of 'truth'- i.e. she is thinking of MORAL truths, and you are thinking of SCIENTIFIC truths. The fact that I can test my beliefs against an objective reality tells me very little about how I should interact with my fellow man.

Anyone who reads a statement like yours who is even a little familiar with Ayn Rand would probably recoil, not at the actual words but at what they fear is coming next- 'reality exists therefore INSERT MORAL CODE HERE'. Unfortunately, we don't have your mother's response, but its possible when she was suggesting totalitarian dangers she was thinking along the lines of objectivism and other MORAL codes.

Comment author: Logos01 23 May 2012 09:51:12PM 0 points [-]

Moral truths which ignore scientific truths are invalid.

View more: Prev | Next