Well, how about this: physics as we know it can be approximated arbitrarily closely by a computable algorithm (and possibly computed directly as well, although I'm less sure about that. Certainly all calculations we can do involving manipulation of symbols are computable). Physics as we know it also seems to be correct to extremely precise degrees anywhere apart from inside a black hole.

Brains are physical things. Now when we consider that thermal noise should have more of an influence than the slight inaccuracy in any computation, what are the chances a brain does anything non-computable that could have any relevance to consciousness? I don't expect to see black holes inside brains, at least.

In any case, your original question was about the moral worth of turing machines, was it not? We can't use "turing machines can't be conscious" as excuse not to worry about those moral questions, because we aren't sure whether turing machines can be conscious. "It doesn't feel like they should be" isn't really a strong enough argument to justify doing something that would result in, for example, the torture of conscious entities if we were incorrect.

So here's my actual answer to your question: as a rule of thumb, act as if any simulation of "sufficient fidelity" is as real as you or I (well, multiplied by your probability that such a simulation would be conscious, maybe 0.5, for expected utilities). This means no killing, no torture, etc.

'Course, this shouldn't be a practical problem for a while yet, and we may have learned more by the time we're creating simulations of "sufficient fidelity".

at this stage, you've just assumed the conclusion. you've just assumed what you want to prove.

No - what I'm pointing out is that the question "what are the ethical implications for turing machines" is the same question as "what are the ethical implications for human beings" in that case.

It's not my job to refute the proposition. Currently, as far as I can tell, the question is open.

Not on Less Wrong, it isn't. But I think I may have misunderstood your situation as being one of somebody coming to Less Wrong to learn about rationality of the "Extreme Bayesian" variety; if you just dropped in here to debate the consciousness question, you probably won't find the experience much fun. ;-)

I did refute it, then my (and several other people's) conjecture would be proven. But if I don't refute it, that doesn't mean your proposition is true, it just means that it hasn't yet been proven false. Those are quite different things, you know.

Less Wrong has different -- and far stricter -- rules of evidence than just about any other venue for such a discussion.

In particular, to meaningfully partake in this discussion, the minimum requirement is to understand the Mind Projection Fallacy at an intuitive level, or else you'll just be arguing about your own intuitions... and everybody will just tune you out.

Without that understanding, you're in exactly the same place as a creationist wandering into an evolutionary biology forum, without understanding what "theory" and "evidence" mean, and expecting everyone to disprove creationism without making you read any introductory material on the subject.

In this case, the introductory material is the Sequences -- especially the ones that debunk supernaturalism, zombies, definitional arguments, and the mind projection fallacy.

When you've absorbed those concepts, you'll understand why the things you're saying are open questions are not even real questions to begin with, let alone propositions to be proved or disproved! (They're actually on a par with creationists' notions of "missing links" -- a confusion about language and categories, rather than an argument about reality.)

I only replied to you because I though perhaps you had read the Sequences (or some portion thereof) and had overlooked their application in this context (something many people do for a while until it clicks that, oh yeah, rationality applies to everything).

So, at this point I'll bow out, as there is little to be gained by discussing something when we can't even be sure we agree on the proper usage of words.