= 27d567bc2d48d9f40e9ccc20ea370b15)
matt1 comments on Rationality Quotes: April 2011 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (384)
No, I was not trying to think along those lines. I must say, I worried in advance that discussing philosophy with people here would be fruitless, but I was lured over by a link, and it seems worse than I feared. In case it isn't clear, I'm perfectly aware what a Turing machine is; incidentally, while I'm not a computer scientist, I am a professional mathematical physicist with a strong interest in computation, so I'm not sitting around saying "OH NOES" while being ignorant of the terms I'm using. I'm trying to highlight one aspect of an issue that appears in many cases: if consciousness (meaning whatever we mean when we say that humans have consciousness) is possible for Turing machines, what are the implications if we do any of the obvious things? (replaying, turning off, etc...) I haven't yet seen any reasonable answer, other than 1) this is too hard for us to work out, but someday perhaps we will understand it (the original answer, and I think a good one in its acknowledgment of ignorance, always a valid answer and a good guide that someone might have thought about things) and 2) some pointless and wrong mocking (your answer, and I think a bad one). edit to add: forgot, of course, to put my current guess as to most likely answer, 3) that consciousness isn't possible for Turing machines.
This is the part where you're going astray, actually. We have no reason to think that human beings are NOT Turing-computable. In other words, human beings almost certainly are Turing machines.
Therefore, consciousness -- whatever we mean when we say that -- is indeed possible for Turing machines.
To refute this proposition, you'd need to present evidence of a human being performing an operation that can't be done by a Turing machine.
Understanding this will help "dissolve" or "un-ask" your question, by removing the incorrect premise (that humans are not Turing machines) that leads you to ask your question.
That is, if you already know that humans are a subset of Turing machines, then it makes no sense to ask what morally justifies treating them differently than the superset, or to try to use this question as a way to justify taking them out of the larger set.
IOW, (the set of humans) is a subset of (the set of turing machines implementing consciousness), which in turn is a proper subset of (the set of turing machines). Obviously, there's a moral issue where the first two subsets are concerned, but not for (the set of turing machines not implementing consciousness).
In addition, there may be some issues as to when and how you're doing the turning off, whether they'll be turned back on, whether consent is involved, etc... but the larger set of "turing machines" is obviously not relevant.
I hope that you actually wanted an answer to your question; if so, this is it.
(In the event you wish to argue for another answer being likely, you'll need to start with some hard evidence that human behavior is NOT being Turing-computable... and that is a tough road to climb. Essentially, you're going to end up in zombie country.)
That's quite easy: I can lift a rock, a Turing machine can't. A Turing machine can only manipulate symbols on a strip of tape, it can't do anything else that's physical.
Your claim that consciousness (whatever we mean when we say that) is possible for Turing machines, rests on the assumption that consciousness is about computation alone, not about computation+some unidentified physical reaction that's absent to pure Turing machines resting in a box on a table.
That consciousness is about computation alone may indeed end up true, but it's as yet unproven.
That sounds like a parody of bad anti-computationalist arguments. To see what's wrong with it, consider the response: "Actually you can't lift a rock either! All you can do is send signals down your spinal column."
What sort of evidence would persuade you one way or the other?
Read the first part of ch.2 of "Good and Real".
Could you clarify why you think that this reading assignment illuminates the question being discussed? I just reread it. For the most part, it is an argument against dualism. It argues that consciousness is (almost certainly) reducible to a physical process.
But this doesn't have anything to do with what ArisKatsaris wrote. He was questioning whether consciousness can be reduced to a purely computational process (without "some unidentified physical reaction that's absent to pure Turing machines".)
Consider the following argument sketch:
Each step above is at least somewhat problematic. Matt1 seemed to be arguing against step 1, and Drescher does respond to that. But ArisKatsaris seemed to be arguing against step 2. My choice would be to expand the definition of 'computation' slightly to include the interactive, asynchronous, and analog, so that I accept step 2 but deny step 3. Over the past decade, Wegner and Goldin have published many papers arguing that computation != TM.
It may well be that you can only get consciousness if you have a non-TM computation (mind) embedded in a system of sensors and actuators (body) which itself interacts with and is embedded in within a (simulated?) real-time environment. That is, when you abstract the real-time interaction away, leaving only a TM computation, you have abstracted away an essential ingredient of consciousness.
It actually sketches what consciousness is and how it works, from which you can see how we could implement something like that as an abstract algorithm.
The value of that description is not so much in reaching a certain conclusion, but in reaching a sense of what exactly are we talking about and consequently why the question of whether "we can implement consciousness as an abstract algorithm" is uninteresting, since at that point you know more about the phenomenon than the words forming the question can access (similarly to how the question of whether crocodile is a reptile is uninteresting, once you know everything you need about crocodiles).
The problem here, I think, is that "consciousness" doesn't get unpacked, and so most of the argument is on the level of connotations. The value of understanding the actual details behind the word, even if just a little bit, is in breaking this predicament.
I think I can see a rube/blegg situation here.
A TM computation perfectly modelling a human brain (let's say) but without any real-time interaction, and a GLUT, represent the two ways in which we can have one of 'intelligent input-output' and 'functional organization isomorphic to that of an intelligent person' without the other.
What people think they mean by 'consciousness' - a kind of 'inner light' which is either present or not - doesn't (straightforwardly) correspond to anything that objectively exists. When we hunt around for objective properties that correlate with places where we think the 'inner light' is shining, we find that there's more than one candidate. Both 'intelligent input-output' and the 'intelligent functional organization' pick out exactly those beings we believe to be conscious - our fellow humans foremost among them. But in the marginal cases where we have one but not the other, I don't think there is a 'further fact' about whether 'real consciousness' is present.
However, we do face the 'further question' of how much moral value to assign in the marginal cases - should we feel guilty about switching off a simulation that no-one is looking at? Should we value a GLUT as an 'end in itself' rather than simply a means to our ends? (The latter question isn't so important given that GLUTs can't exist in practice.)
I wonder if our intuition that the physical facts underdetermine the answers to the moral questions is in some way responsible for the intuition of a mysterious non-physical 'extra fact' of whether so-and-so is conscious. Perhaps not, but there's definitely a connection.
Yes, and I did not even attempt to address that 'further question' because it seems to me that that question is at least an order of magnitude more confused than the relatively simple question about consciousness.
But, if I were to attempt to address it, I would begin with the lesson from Econ 101 that dissolves the question "What is the value of item X?". The dissolution begins by requesting the clarifications "Value to whom?" and "Valuable in what context?" So, armed with this analogy, I would ask some questions:
My example was one where (i) the 'whom' and the 'context' are clear and yet (ii) this obviously doesn't dissolve the problem.
It may be a step toward dissolving the problem. It suggests the questions:
My own opinion, which I won't try to convince you of, is that there is no moral significance without interaction and reciprocity (in a fairly broad sense of those two words.)
It does, to some extent. There is a simple description that moves the discussion further. Namely, consciousness is a sensory modality that observes its own operation, and as a result it also observes itself observing its own operation, and so on; as well as observing external input, observing itself observing external input, and so on; and observing itself determining external output, etc.
This is an important idea, but I don't think it can rescue the everyday intuition of the "inner light".
I can readily imagine an instantiation of your sort of "consciousness" in a simple AI program of the kind we can already write. No doubt it would be an interesting project, but mere self-representation (even recursive self-representation) wouldn't convince us that there's "something it's like" to be the AI. (Assume that the representations are fairly simple, and the AI is manipulating them in some fairly trivial way.)
Conversely, we think that very young children and animals are conscious in the "inner light" sense, even though we tend not to think of them as "recursively observing themselves". (I have no idea whether and in what sense they actually do. I also don't think "X is conscious" is unambiguously true or false in these cases.)
I think you're trivializing the issue. A Turing machine is an abstraction, it isn't a real thing. The claim that a human being is a Turing machine means that, in the abstract, a certain aspect of human beings can be modeled as a Turing machine. Conceptually, it might be the case, for instance, that the universe itself can be modeled as a Turing machine, in which case it is true that a Turing machine can lift a rock.
So... you support euthanasia for quadriplegics, then, or anyone else who can't pick up a rock? Or people who are so crippled they can only communicate by reading and writing braille on a tape, and rely on other human beings to feed them and take care of them?
This "unidentified physical reaction" would also need to not be turing-computable to have any relevance. Otherwise, you're just putting forth another zombie-world argument.
At this point, we have no empirical reason to think that this unidentified mysterious something has any existence at all, outside of a mere intuitive feeling that it "must" be so.
And so, all we have are thought experiments that rest on using slippery word definitions to hide where the questions are being begged, presented as intellectual justification for these vague intuitions... like arguments for why the world must be flat or the sun must go around the earth, because it so strongly looks and feels that way.
(IOW, people try to prove that their intuitions or opinions must have some sort of physical form, because those intuitions "feel real". The error arises from concluding that the physical manifestation must therefore exist "out there" in the world, rather than in their own brains.)
A zombie-world seems extremely improbable to have evolved naturally, (evolved creatures coincidentally speaking about their consciousness without actually being conscious), but I don't see why a zombie-world couldn't be simulated by a programmer who studied how to compute the effects of consciousness, without actually needing to have the phenomenon of consciousness itself.
The same way you don't need to have an actual solar system inside your computer, in order to compute the orbits of the planets -- but it'd be very unlikely to have accidentally computed them correctly if you hadn't studied the actual solar system.
Do you have any empirical reason to think that consciousness is about computation alone? To claim Occam's razor on this is far from obvious, as the only examples of consciousness (or talking about consciousness) currently concern a certain species of evolved primate with a complex brain, and some trillions of neurons, all of which have have chemical and electrical effects, they aren't just doing computations on an abstract mathematical universe sans context.
Unless you assume the whole universe is pure mathematics, so there's no difference between the simulation of a thing and the thing itself. Which means there's no difference between the mathematical model of a thing and the thing itself. Which means the map is the territory. Which means Tegmark IV.
And Tegmark IV is likewise just a possibility, not a proven thing.
This is a "does the tree make a sound if there's no-one there to hear it?" argument.
That is, it assumes that there is a difference between "effects of consciousness" and "consciousness itself" -- in the same way that a connection is implied between "hearing" and "sound".
That is, the argument hinges on the definition of the word whose definition is being questioned, and is an excellent example of intuitions feeling real.
Not quite. What I'm saying is there might be a difference between the computation of a thing and the thing itself. It's basically an argument against the inevitability of Tegmark IV.
A Turing machine can certainly compute everything there is to know about lifting rocks and their effects -- but it still can't lift a rock. Likewise a Turing machine could perhaps compute everything there was to know about consciousness and its effects -- but perhaps it still couldn't actually produce one.
Or at least I've not been convinced that it's a logical impossibility for it to be otherwise; nor that I should consider it my preferred possibility that consciousness is solely computation, nothing else.
Wouldn't the same reasoning mean that all physical processes have to be solely computation? So it's not just "a Turing machine can produce consciousness", but "a Turing machine can produce a new physical universe", and therefore "Yeah, Turing Machines can lift real rocks, though it's real rocks in a subordinate real universe, not in ours".
I think you mean, it's the skeleton of an argument you could advance if there turned out to actually be some meaning to the phrase "difference between the computation of a thing and the thing itself".
Herein lies the error: it's not up to anybody else to convince you it's logically impossible, it's up to you to show that you're even describing something coherent in the first place.
Really, this is another LW-solved philosophical problem; you just have to grok the quantum physics sequence, in addition to the meanings-of-words one: when you understand that physics itself is a machine, it dissolves the question of what "simulation" or "computation" mean in this context. That is, you'll realize that the only reason you can even ask the question is because you're confusing the labels in your mind with real things.
Could you point to the concrete articles that supposedly dissolve this question? I find the question of what "computation" means as still very much open, and the source of a whole lot of confusion. This is best seen when people attempt to define what constitutes "real" computation as opposed to mere table lookups, replays, state machines implemented by random physical processes, etc.
Needless to say, this situation doesn't give one the license to jump into mysticism triumphantly. However, as I noted in a recent thread, I observe an unpleasant tendency on LW to use the notions of "computation," "algorithms," etc. as semantic stop signs, considering how ill-understood they presently are.
Please note that I did not say the sequence explains "computation"; merely that it dissolves the illusion the intuitive notion of a meaningful distinction between a "computation" or "simulation" and "reality".
In particular, an intuitive understanding that people are made of interchangeable particles and nothing else, dissolves the question of "what happens if somebody makes a simulation of you?" in the same way that it dissolves "what happens if there are two copies of you... which one's the real one?"
That is, the intuitive notion that there's something "special" about the "original" or "un-simulated" you is incoherent, because the identity of entities is an unreal concept existing only in human brains' representation of reality, rather than in reality itself.
The QM sequence demonstrates this; it does not, AFAIR, attempt to rigorously define "computation", however.
Those sound like similarly confused notions to me -- i.e., tree-sound-hearing questions, rather than meaningful ones. I would therefore refer such questions to the "usage of words" sequence, especially "How an Algorithm Feels From The Inside" (which was my personal source of intuitions about such confusions).
Here's what I think. It's just a "mysterious answer to a mysterious question" but it's the best I can come up with.
From the perspective of a simulated person, they are conscious. A 'perspective' is defined by a mapping of certain properties of the simulated person to abstract, non-uniquely determined 'mental properties'.
Perspectives and mental properties do not exist (that's the whole point - they're subjective!) It's a category mistake to ask: does this thing have a perspective? Things don't "have" perspectives the way they have position or mass. All we can ask is: "From this perspective (which might even be the perspective of a thermostat), how does the world look?"
The difference between a person in a simulation and a 'real person' is that defining the perspective of a real person is slightly 'easier', slightly 'more natural'. But if the simulated and real versions are 'functionally isomorphic' then any perspective we assign to one can be mapped onto the other in a canonical way. (And having pointed these two facts out, we thereby exhaust everything there is to be said about whether simulated people are 'really conscious'.)
ETA: I'm actually really interested to know what the downvoter thinks. I mean, I know these ideas are absurd but I can't see any other way to piece it together. To clarify: what I'm trying to do is take the everyday concept of "what it's likeness" as far as it will go without either (a) committing myself to a bunch of arbitrary extra facts (such as 'the exact moment when a person first becomes conscious' and 'facts of the matter' about whether ants/lizards/mice/etc are conscious) or (b) ditching it in favour of a wholly 'third person' Dennettian notion of consciousness. (If the criticism is simply that I ought to ditch it in favour of Dennett-style consciousness then I have no reply (ultimately I agree!) but you're kind-of missing the point of the exercise.)
thanks. my point exactly.
You wrote: "This is the part where you're going astray, actually. We have no reason to think that human beings are NOT Turing-computable. In other words, human beings almost certainly are Turing machines."
at this stage, you've just assumed the conclusion. you've just assumed what you want to prove.
"Therefore, consciousness -- whatever we mean when we say that -- is indeed possible for Turing machines."
having assumed that A is true, it is easy to prove that A is true. You haven't given an argument.
"To refute this proposition, you'd need to present evidence of a human being performing an operation that can't be done by a Turing machine."
It's not my job to refute the proposition. Currently, as far as I can tell, the question is open. If I did refute it, then my (and several other people's) conjecture would be proven. But if I don't refute it, that doesn't mean your proposition is true, it just means that it hasn't yet been proven false. Those are quite different things, you know.
Well, how about this: physics as we know it can be approximated arbitrarily closely by a computable algorithm (and possibly computed directly as well, although I'm less sure about that. Certainly all calculations we can do involving manipulation of symbols are computable). Physics as we know it also seems to be correct to extremely precise degrees anywhere apart from inside a black hole.
Brains are physical things. Now when we consider that thermal noise should have more of an influence than the slight inaccuracy in any computation, what are the chances a brain does anything non-computable that could have any relevance to consciousness? I don't expect to see black holes inside brains, at least.
In any case, your original question was about the moral worth of turing machines, was it not? We can't use "turing machines can't be conscious" as excuse not to worry about those moral questions, because we aren't sure whether turing machines can be conscious. "It doesn't feel like they should be" isn't really a strong enough argument to justify doing something that would result in, for example, the torture of conscious entities if we were incorrect.
So here's my actual answer to your question: as a rule of thumb, act as if any simulation of "sufficient fidelity" is as real as you or I (well, multiplied by your probability that such a simulation would be conscious, maybe 0.5, for expected utilities). This means no killing, no torture, etc.
'Course, this shouldn't be a practical problem for a while yet, and we may have learned more by the time we're creating simulations of "sufficient fidelity".
No - what I'm pointing out is that the question "what are the ethical implications for turing machines" is the same question as "what are the ethical implications for human beings" in that case.
Not on Less Wrong, it isn't. But I think I may have misunderstood your situation as being one of somebody coming to Less Wrong to learn about rationality of the "Extreme Bayesian" variety; if you just dropped in here to debate the consciousness question, you probably won't find the experience much fun. ;-)
Less Wrong has different -- and far stricter -- rules of evidence than just about any other venue for such a discussion.
In particular, to meaningfully partake in this discussion, the minimum requirement is to understand the Mind Projection Fallacy at an intuitive level, or else you'll just be arguing about your own intuitions... and everybody will just tune you out.
Without that understanding, you're in exactly the same place as a creationist wandering into an evolutionary biology forum, without understanding what "theory" and "evidence" mean, and expecting everyone to disprove creationism without making you read any introductory material on the subject.
In this case, the introductory material is the Sequences -- especially the ones that debunk supernaturalism, zombies, definitional arguments, and the mind projection fallacy.
When you've absorbed those concepts, you'll understand why the things you're saying are open questions are not even real questions to begin with, let alone propositions to be proved or disproved! (They're actually on a par with creationists' notions of "missing links" -- a confusion about language and categories, rather than an argument about reality.)
I only replied to you because I though perhaps you had read the Sequences (or some portion thereof) and had overlooked their application in this context (something many people do for a while until it clicks that, oh yeah, rationality applies to everything).
So, at this point I'll bow out, as there is little to be gained by discussing something when we can't even be sure we agree on the proper usage of words.
Can you expand on why you expect human moral intuition to give reasonably clear answers when applied to situations involving conscious machines ?
Another option:
it's morally acceptable to terminate a conscious program if it wants to be terminated
it's morally questionable(wrong, but to lesser degree) to terminate a conscious program against its will if it is also possible to resume execution
it is horribly wrong to turn off a conscious program against its will if it cannot be resumed(murder fits this description currently)
performing other operations on the program that it desires would likely be morally acceptable, unless the changes are socially unacceptable
performing other operations on the program against its will is morally unacceptable to a variable degree (brainwashing fits in this category)
These seem rather intuitive to me, and for the most part I just extrapolated from what it is moral to do to a human. Conscious program refers here to one running on any system, including wetware, such that these apply to humans as well. I should note that I am in favor of euthanasia in many cases, in case that part causes confusion.
btw, I'm fully aware that I'm not asking original questions or having any truly new thoughts about this problem. I just hoped maybe someone would try to answer these old questions given that they had such confidence in their beliefs.
If you think 1 is the correct answer, you should be aware that this website is for people who do not wait patiently for a someday where we might have an understanding. One of the key teachings of this website is to reach out and grab an understanding with your own two hands. And you might add a 4 to that list, "death threats", which does not strike me as the play either.
You should be aware that in many cases, the sensible way to proceed is to be aware of the limits of your knowledge. Since the website preaches rationality, it's worth not assigning probabilities of 0% or 100% to things which you really don't know to be true or false. (btw, I didn't say 1) is the right answer, I think it's reasonable, but I think it's 3) )
And sometimes you do have to wait for an answer. For a lesson from math, consider that Fermat had flat out no hope of proving his "last theorem", and it required a couple hundred years of apparently unrelated developments to get there....one could easily give a few hundred examples of that sort of thing in any hard science which has a long enough history.
Uh I believe you will find that Fermat in fact had a truly marvelous proof of his last theorem? The only thing he was waiting on was the invention of a wider margin.
Little-known non-fact: there were wider margins available at the time, but it was not considered socially acceptable to use them for accurate proofs, or more generally for true statements at all; they were merely wide margins for error.
I wonder how much the fame of Fermat's Last Theorem is due to the fact that, (a) he claimed to have found a proof, and (b) nobody was able to prove it. Had he merely stated it as a conjecture without claiming that he had proven it, would anywhere near the same effort have been put into proving it?
Almost certainly not. A lot of the historical interest came precisely because he claimed to have a proof. In fact, there were a fair number of occasions where he claimed to have a proof and a decent chunk of number theory in the 1700s and early 1800s was finding proofs for the statements that Fermat had said he had a proof for. It was called "Fermat's Last Theorem" because it was the last one standing of all his claims.