at this stage, you've just assumed the conclusion. you've just assumed what you want to prove.

No - what I'm pointing out is that the question "what are the ethical implications for turing machines" is the same question as "what are the ethical implications for human beings" in that case.

It's not my job to refute the proposition. Currently, as far as I can tell, the question is open.

Not on Less Wrong, it isn't. But I think I may have misunderstood your situation as being one of somebody coming to Less Wrong to learn about rationality of the "Extreme Bayesian" variety; if you just dropped in here to debate the consciousness question, you probably won't find the experience much fun. ;-)

I did refute it, then my (and several other people's) conjecture would be proven. But if I don't refute it, that doesn't mean your proposition is true, it just means that it hasn't yet been proven false. Those are quite different things, you know.

Less Wrong has different -- and far stricter -- rules of evidence than just about any other venue for such a discussion.

In particular, to meaningfully partake in this discussion, the minimum requirement is to understand the Mind Projection Fallacy at an intuitive level, or else you'll just be arguing about your own intuitions... and everybody will just tune you out.

Without that understanding, you're in exactly the same place as a creationist wandering into an evolutionary biology forum, without understanding what "theory" and "evidence" mean, and expecting everyone to disprove creationism without making you read any introductory material on the subject.

In this case, the introductory material is the Sequences -- especially the ones that debunk supernaturalism, zombies, definitional arguments, and the mind projection fallacy.

When you've absorbed those concepts, you'll understand why the things you're saying are open questions are not even real questions to begin with, let alone propositions to be proved or disproved! (They're actually on a par with creationists' notions of "missing links" -- a confusion about language and categories, rather than an argument about reality.)

I only replied to you because I though perhaps you had read the Sequences (or some portion thereof) and had overlooked their application in this context (something many people do for a while until it clicks that, oh yeah, rationality applies to everything).

So, at this point I'll bow out, as there is little to be gained by discussing something when we can't even be sure we agree on the proper usage of words.

To refute this proposition, you'd need to present evidence of a human being performing an operation that can't be done by a Turing machine.

That's quite easy: I can lift a rock, a Turing machine can't. A Turing machine can only manipulate symbols on a strip of tape, it can't do anything else that's physical.

Your claim that consciousness (whatever we mean when we say that) is possible for Turing machines, rests on the assumption that consciousness is about computation alone, not about computation+some unidentified physical reaction that's absent to pure Turing machines resting in a box on a table.

That consciousness is about computation alone may indeed end up true, but it's as yet unproven.

If you think 1 is the correct answer, you should be aware that this website is for people who do not wait patiently for a someday where we might have an understanding. One of the key teachings of this website is to reach out and grab an understanding with your own two hands. And you might add a 4 to that list, "death threats", which does not strike me as the play either.

if consciousness (meaning whatever we mean when we say that humans have consciousness) is possible for Turing machines,

This is the part where you're going astray, actually. We have no reason to think that human beings are NOT Turing-computable. In other words, human beings almost certainly are Turing machines.

Therefore, consciousness -- whatever we mean when we say that -- is indeed possible for Turing machines.

To refute this proposition, you'd need to present evidence of a human being performing an operation that can't be done by a Turing machine.

Understanding this will help "dissolve" or "un-ask" your question, by removing the incorrect premise (that humans are not Turing machines) that leads you to ask your question.

That is, if you already know that humans are a subset of Turing machines, then it makes no sense to ask what morally justifies treating them differently than the superset, or to try to use this question as a way to justify taking them out of the larger set.

IOW, (the set of humans) is a subset of (the set of turing machines implementing consciousness), which in turn is a proper subset of (the set of turing machines). Obviously, there's a moral issue where the first two subsets are concerned, but not for (the set of turing machines not implementing consciousness).

In addition, there may be some issues as to when and how you're doing the turning off, whether they'll be turned back on, whether consent is involved, etc... but the larger set of "turing machines" is obviously not relevant.

I hope that you actually wanted an answer to your question; if so, this is it.

(In the event you wish to argue for another answer being likely, you'll need to start with some hard evidence that human behavior is NOT being Turing-computable... and that is a tough road to climb. Essentially, you're going to end up in zombie country.)

What's wrong with Dennett's explanation of consciousness?

No, I was not trying to think along those lines. I must say, I worried in advance that discussing philosophy with people here would be fruitless, but I was lured over by a link, and it seems worse than I feared. In case it isn't clear, I'm perfectly aware what a Turing machine is; incidentally, while I'm not a computer scientist, I am a professional mathematical physicist with a strong interest in computation, so I'm not sitting around saying "OH NOES" while being ignorant of the terms I'm using. I'm trying to highlight one aspect of an issue that appears in many cases: if consciousness (meaning whatever we mean when we say that humans have consciousness) is possible for Turing machines, what are the implications if we do any of the obvious things? (replaying, turning off, etc...) I haven't yet seen any reasonable answer, other than 1) this is too hard for us to work out, but someday perhaps we will understand it (the original answer, and I think a good one in its acknowledgment of ignorance, always a valid answer and a good guide that someone might have thought about things) and 2) some pointless and wrong mocking (your answer, and I think a bad one). edit to add: forgot, of course, to put my current guess as to most likely answer, 3) that consciousness isn't possible for Turing machines.

This is a fair answer. I disagree with it, but it is fair in the sense that it admits ignorance. The two distinct points of view are that (mine) there is something about human consciousness that cannot be explained within the language of Turing machines and (yours) there is something about human consciousness that we are not currently able to explain in terms of Turing machines. Both people at least admit that consciousness has no explanation currently, and absent future discoveries I don't think there is a sure way to tell which one is right.

I find it hard to fully develop a theory of morality consistent with your point of view. For example, would it be wrong to (given a computer simulation of a human mind) run that simulation through a given painful experience over and over again? Let us assume that the painful experience has happened once...I just ask whether it would be wrong to rerun that experience. After all, it is just repeating the same deterministic actions on the computer, so nothing seems to be wrong about this. Or, for example, if I make a backup copy of such a program, and then allow that backup to run for a short period of time under slightly different stimuli, at which point does that copy acquire an existence of its own, that would make it wrong to delete that copy in favor of the original? I could give many other similar questions, and my point is not that your point of view denies a morality, but rather that I find it hard to develop a full theory of morality that is internally consistent and that matches your assumptions (not that developing a full theory of morality under my assumptions is that much easier).

Among professional scientists and mathematicians, I have encountered both viewpoints: those who hold it obvious to anyone with even the simplest knowledge that Turing machines cannot be conscious, and those who hold that the opposite it true. Mathematicians seem to lean a little more toward the first viewpoint than other disciplines, but it is a mistake to think that a professional, world-class research level, knowledge of physics, neuroscience, mathematics, or computer science necessarily inclines one towards the soulless viewpoint.

Hmm, I don't happen to find your argument very convincing. I mean, what it does is to pay attention to some aspect of the original mistaken statement, then find another instance sharing that aspect which is transparently ridiculous.

But is this sufficient? You can model the statement "apples and oranges are good fruits" in predicate logic as "for all x, Apple(x) or Orange(x) implies Good(x)" or in propositional logic as "A and O" or even just "Z". But it should really depend on what aspect of the original statement you want to get at. You want a model which captures precisely those aspects you want to work with.

So your various variables actually confused the hell outta me there. I was trying to match them up with the original statement and your reductio example. All the while not really understanding which was relevant to the confusion. It wasn't a pleasant experience :(

It seems to me much simpler to simply answer: "Turing machine-ness has no bearing on moral worth". This I think gets straight to the heart of the matter, and isolates clearly the confusion in the original statement.

Or further guess at the source of the confusion, the person was trying to think along the lines of: "Turing machines, hmm, they look like machines to me, so all Turing machines are just machines, like a sewing machine, or my watch. Hmm, so humans are Turing machines, but by my previous reasoning this implies humans are machines. And hmm, furthermore, machines don't have moral worth... So humans don't have moral worth! OH NOES!!!"

Your argument seems like one of those long math proofs which I can follow step by step but cannot grasp its overall structure or strategy. Needless to say, such proofs aren't usually very intuitively convincing.

(but I could be generalizing from one example here)

If you think that humans are nothing but Turing machines, why is it morally wrong to kill a person but not morally wrong to turn off a computer?

Your question has the form:

If A is nothing but B, then why is it X to do Y to A but not to do Y to C which is also nothing but B?

This following question also has this form:

If apple pie is nothing but atoms, why is it safe to eat apple pie but not to eat napalm which is also nothing but atoms?

And here's the general answer to that question: the molecules which make up apple pie are safe to eat, and the molecules which make up napalm are unsafe to eat. This is possible because these are not the same molecules.

Now let's turn to your own question and give a general answer to it: it is morally wrong to shut off the program which makes up a human, but not morally wrong to shut off the programs which are found in an actual computer today. This is possible because these are not the same programs.

At this point I'm sure you will want to ask: what is so special about the program which makes up a human, that it would be morally wrong to shut off the program? And I have no answer for that. Similarly, I couldn't answer you if you asked me why the molecules of apple pie are safe to eat and the those of napalm are not.

As it happens, chemistry and biology have probably advanced to the point at which the question about apple pie can be answered. However, the study of mind/brain is still in its infancy, and as far as I know, we have not advanced to the equivalent point. But this doesn't mean that there isn't an answer.

This is, of course, pretty much the right answer to anyone who asserts that without God, they could just kill anyone they wanted.