All of poke's Comments + Replies

Most transhumanist ideas fall under the category of "not even wrong." Drexler's Nanosystems is ignored because it's a work of "speculative engineering" that doesn't address any of the questions a chemist would pose (i.e., regarding synthesis). It's a non-event. It shows that you can make fancy molecular structures under certain computational models. SI is similar. What do you expect a scientist to say about SI? Sure, they can't disprove the notion, but there's nothing for them to discuss either. The transhumanist community has a tendenc... (read more)

It's interesting that you mention Rodney Brooks. I've always found his work poorly written and lacking in clarity despite being sympathetic to his views. He must come across better in person. As Shane points out though, Brooks' work has the rare quality in AI that it is productive and has found widespread application in industry.

As for the Venture Capitalists, I don't find it surprising that Silicon Valley VCs share some of your interests. It's like discovering that software engineers share an interest in AD&D and collectibles. All these guys are enthu... (read more)

mtraven,

Most sane and intelligent people with religious tendencies (and there are many, although they don't seem to get much press) understand that if "god" means anything, it is a pointer towards something unknown and perhaps unknowable, and arguing about whether it exists in the physical sense is missing the point completely.

This is just a version of my second option available to the theist. There's a knowable "physical" world and an unknowable one beyond it. There's no reason to believe this is the case. Moreover, if you believed ... (read more)

This is why I claim that atheism is an established scientific result. One of the strongest lines of evidence is, indeed, that we have successfully reduced minds and shown the notion of an irreducible mind to be incoherent. Mind as an irreducible simple is basic to all monotheistic religions. Demonstrating something once thought coherent to be incoherent is, of course, one of the strongest lines of evidence in science. Other avenues through which atheism has been established by science include conservation in physics, chemistry and biology (which led direct... (read more)

It's true that contemporary philosophy is still very much obsessed with language despite attempts by practioners to move on. Observation is talked about in terms of observation sentences. Science is taken to be a set of statements. Realism is taken to be the doctrine that there are objects to which our statements refer. Reductionism is the ability to translate a sentence in one field into a sentence in another. The philosophy of mind concerns itself with finding a way to reconcile the lack of sentence-like structures in our brain with a perverse desire for... (read more)

If you look through a microscope you'll notice the only major difference between the nervous system and other tissues is that the nervous system exhibits network connectivity. Cells in tissues are usually arranged in such a way that they only connect to their nearest neighbor. Many tissues exhibit electrical activity, communication between cells, coordinated activity, etc, in the same way as neurons. If networks of neurons can be said to be performing computations then so can other tissues. I'm not familar with the biology of trees but I don't see why they couldn't be said to be 'thinking' if we're going to equate thinking with computation.

This demonstrates quite nicely the problem with the magical notion of an "internal representation." (Actually, there's two magical notions, since both "internal" and "representation" are separately magical.) You could easily replace "internal representation" with "soul" in this essay and you'd get back the orthodox thinking about humans and animals of the last two thousand years. Given that there is both no evidence nor any chance of evidence either for "internal representations" or "souls" and neither is well-defined (or defined at all), you might as well go ahead and make the substitution. This entire essay is pure mysticism.

It's not clear that you're a verificationist but you're clearly an Empiricist. I think that's problematic. Unless you believe something magical happens at the retina, then there's no more reason to privilege what happens at the retina or in the brain, than there is the wire connecting the dial to the voltmeter. It's all causal linkage. We can use the same standards of reliability for people as we do wires. The sensory periphery is just not particularly interesting.

Richard Kennaway,

Whatever views his belief may be compatible with, it is not compatible with reality. That is, it is false. Intentionality is explainable in physical terms. An intention is the reference signal of a control system, and a control system is something that acts so as to maintain a perceptual signal close to its reference signal.

Everybody has a pet theory of intentionality. The problem isn't intentionality but why anybody would want to explain intentionality in the first place. You're either explaining: (a) a part of what you take to be your... (read more)

Methodological behaviorism took private mental events to be off limits but (most) behaviorists still believed they existed. Skinner took introspection and self-knowledge to by types of behavior and explicitly denied the mental. Eliezer's analysis is correct insofar as Skinner denied the mental but the passages about not being able to account for complex behavior are wrong. Skinner took behavior to be a product of environmental conditioning and evolved physiology.

Here's Skinner explaining radical behaviorism in the opening of About Behaviorism:

"Mentali... (read more)

Skinner was correct that mind, intentionality, thought, desire, etc, are unscientific. Where behaviorism went wrong was ascribing behavior to conditioning and underplaying the role of biology (although Skinner never denied the importance of biology; unlike Chomsky and the computationalists). I'd accuse computationalism of being "cryptodualism" except that Chomsky's project was explicitly Cartesian and was only non-dualistic in the sense that he believed the laws of physics would have to change to incorporate non-biological computational models of... (read more)

I wouldn't want to be a wirehead. I do things like exercise to keep my mood up now but I think of it terms of wanting to be productive rather than happy. (I find that exercise, health and regular sleep/wake cycles are essential for this.) If you could wire me up to be smarter and more productive (intellectually), but the cost was chronic pain, I'd probably sign up for that. (I can't really imagine how you could reconcile higher productivity with chronic pain though; the experience of pain seems to necessarily involve restricted attention.)

Andy Wood,

I'm just curious - was the despair about anything? Did it have no referent at all? You had a stable environment, good relationship with parents, self-confidence, social success, and yet still despaired? Was there no consistent content in your despairing thoughts?

I had all those things. Before I became depressed I stopped being sociable and started having problems with school attendance; I don't know if that was the cause of my depression or just an early development of it. I was certainly very bored at school and my home environment didn't off... (read more)

I've suffered from clinical depression with absolutely zero correlation to social factors and life circumstances. Between onset at age 11 and my early 20s I experience pervasive, uninterrupted despair. Oddly enough, it never affected my goals or terminal values, just my ability to achieve them. Then again, many people (perhaps the majority) die with many of the same goals they had in their youth, having done absolutely no work toward achieving them; so I'm not convinced explicitly held goals have a strong causal relation to behavior; perhaps having a goal ... (read more)

What could be more exciting than embracing nihilism?

"Should" has obvious non-moral uses: you should open the door before attempting to walk through it. "Right" and "better" too: you need the right screwdriver; it's better to use a torque driver. We can use these words in non-problematic physical situations. I think this makes it obvious that morality is in most cases just a supernatural way of talking about consequences. "You shouldn't murder your rival" implies that there will be negative consequences to murdering your rival. If you ask the average person they'll eve... (read more)

As I said previously, I think "moral progress" is the heroic story we tell of social change, and I find it unlikely that these changes are really caused by moral deliberation. I'm not a cultural relativist but I think we need to be more attuned to the fact that people inside a culture are less harmed by its practices than outsiders feel they would be in that culture. You can't simply imagine how you would feel as, say, a woman in Islam. Baselines change, expectations change, and we need to keep track of these things.

As for democracy, I think ther... (read more)

I remember first having this revelation as something along the lines of: "You know when you're in love or overcome by anger, and you do stupid things, and afterward you wonder what the hell you were thinking? Well, your 'normal' emotional states are just like that, except you never get that moment of reflection to wonder what the hell you were thinking." I tried to resolve it with the kind of reflective deliberation that I think you're prescribing here. Later I adopted a sort of happy fatalism: We're trapped inside our own psychology and that's f... (read more)

Unknown and Hopefully Anonymous, If basing your beliefs on established science and systematically rejecting every incompatible methodology is "religion" then stick a ridiculous hat on my head and call me the Pope of Reality.

I don't buy this sort of skepticism at all. Yes, we can imagine that the external world in an illusion, but the basic flaw is (like so much in philosophy) privileging our ability to imagine something over science. Whether we can be deceived in this way is an empirical matter. Yes, you can say "everything you learned about empirical science is part of the illusion," but all you've done is taken your ability to imagine an outcome and privileged that above scientific experiment. Science always trumps imagination. It is therefore, I think, impossible... (read more)

Douglas Knight, I'm not sure what predictions you're referring to. Statistical methods have a good pedigree. I take a correlation to be a correlation and try not to overinterpret it.

I'm very strict about this. I only accept claims that come out of science. I have a narrow definition of science based on lineage: you have to be able to trace it back to settled physics. Physics, chemistry, biochemistry, biology, molecular biology, neural biology, etc, all have strict lines of descent. Much of theoretical psychology, on the other hand (to give an example), does not; it's ab initio theorizing. Anything that is not science (so narrowly defined) I take to be noise. Systematic and flagrant abuse of the "genetic fallacy" is probably the quickest way to truth.

I think the best way to display the sheer mind-boggling absurdity of the "problem of induction" is to consider that we have two laws: the first law is the law science gives us for the evolution of a system and the second law simply states that the first law holds until time t and then "something else" happens. The first law is a product of the scientific method and the second law conforms to our intuition of what could happen. What the problem of induction is actually saying is that imagination trumps science. That's ridiculous. It's ap... (read more)

If somebody said to me "morality is just what we do." If they presented evidence that the whole apparatus of their moral philosophy was a coherent description of some subset of human psychology and sociology. Then that would be enough for me. It's just a description of a physical system. Human morality would be what human animals do. Moral responsibility wouldn't be problematic; moral responsibility could be as physical as gravity if it were psychologically and sociologically real. "I have a moral responsibility" would be akin to "... (read more)

My response to these questions is simply this: Once the neurobiology, sociology and economics is in, these questions will either turn out to have answers or to be the wrong questions (the latter possibility being the much more probable outcome). The only one I know how to answer is the following:

Do the concepts of "moral error" and "moral progress" have referents?

The answer being: Probably not. Reality doesn't much care for our ways of speaking.

A longer (more speculative) answer: The situation changes and we come up with a moral stor... (read more)

This dialogue leads me to conclude that "fairness" is a form of social lubricant that ensures our pies don't get cold while we're busy arguing. The meta-rule for fairness rules would then be: (1) fast; (2) easy to apply; and (3) everybody gets a share.

(1) Buy a country. You could probably bribe your way into becoming dictator of North Korea or Myanmar or somewhere similar.

(2) Build a huge army.

(3) Crash the US economy.

(4) Take over the world.

(5) Profit.

You can fully describe the mind/brain in terms of dynamics without reference to logic or data. But you can't do the reverse. I maintain that the dynamics are all that matters and the rest is just folk theory tarted up with a bad analogy (computationalism).

Unknown,

For all those who have said that morality makes no difference to them, I have another question: if you had the ring of Gyges (a ring of invisibility) would that make any difference to your behavior?

Sure. I could get away with doing all sorts of things. No doubt the initial novelty and power rush would cause me to do some things that would be quite perverted and that I'd feel guilty about. I don't think that's the same as a world without morality though. You seem to view morality as a constraint whereas I view it as a folk theory that describes a subset of human behavior. (I take Eliezer to mean that we're rejecting morality at an intellectual level rather than rewiring our brains.)

I'd do everything I do now. You can't escape your own psychology and I've already expressed my skepticism about the efficacy of moral deliberation. I'll go further and say that nobody would act any differently. Sure, after you shout in from the rooftops, maybe there will be an upsurge in crime and the demand for black nail polish for a month or so but when the dust settled nothing would have changed. People would still cringe at the sight of blood and still react to the pain of others just as they react to their own pain. People would still experience guil... (read more)

I think you have to be careful when you say,

trying to use your brain to understand something that is not like your brain.

We can't use our brains to understand brains that are like our brains. We don't have that kind of access. Empathy is a function and not something you just get for free on account of similarity. Where we have obvious faculties in this area - understanding the emotional state of another person - I don't see any strong differences between same sex and opposite sex empathy. We can all tell when a member of the opposite sex is distressed; ... (read more)

So is the reason I should believe this space of minds-in-general exists at all going to come in a later post?

I can certainly agree that you rely on this sort of reasoning a lot. But I don't think what you do is much of an improvement over what you're criticizing. You just take words and make "surface analogies" with "cognitive algorithms." The useful thing about these "cognitive algorithms" is that, being descriptions of "deep causes" (whatever those are) rather than anything we know to actually exist in the world (like, say, neurons), you can make them do whatever you please with total disregard for reality.

Saying that a n... (read more)

You essentially posit a "decision algorithm" to which you ascribe the sensations most people attribute to free will. I don't think this is helpful and it seems like a cop-out to me. What if the way the brain makes decisions doesn't translate well onto the philosophical apparatus of possibility and choice? You're just trading "suggestively named LISP tokens" for suggestively named algorithms. But even if the brain does do something we could gloss in technical language as "making choices among possibilities" there still aren't r... (read more)

michael vassar,

I think you misunderstand me. I'm not being cynical; I'm trying to demonstrate that moral dilemmas and moral deliberation aren't empirically established. I tried to do this, first, by pointing out that what most people consider the subject of morality differs substantially from the subject of academic philosophers and, second, by arguing that the type of moral reasoning found in philosophy isn't found in society at large and doesn't influence it. People really do heroically rescue orphans from burning buildings in real life and they do it wi... (read more)

The type of possibility you describe is just a product of our ignorance about our own or others psychology. If I don't understand celestial mechanics I might claim that Mars could be anywhere in its orbit at any time. If somebody then came along and taught me celestial mechanics I could then argue that Mars could still be anywhere if it wanted to. This is just saying that Mars could be anywhere if Mars was different. It gets you exactly nothing.

michael vassar,

I'm skeptical as to whether the affirmed moralities play a causal role in their behavior. I don't think this is obvious. Cultures that differ in what we call moral behavior also differ in culinary tastes but we don't think one causes the other; it's possible that they have their behaviors and they have their explanations of their behaviors and the two do not coincide (just as astrology doesn't coincide with astronomy). I'm also therefore skeptical that changes over time are caused by moral deliberation; obviously if morality plays no causal ... (read more)

I agree that determinism doesn't undermine morality in the way you describe. I remain, however, a moral skeptic (or, perhaps more accurately, a moral eliminativist). I'm skeptical that moral dilemmas exist outside of thought experiments and the pages of philosophy books and I'm skeptical that moral deliberation achieves anything. Since people are bound to play out their own psychology, and since we're inherently social animals and exist in a social environment, I find it unlikely that people would behave substantially different if we eliminated "moral... (read more)

I picked up an intuitive sense that real thinking was that which could force you into an answer whether you liked it or not, and fake thinking was that which could argue for anything.

This is very dangerous. I think a great example of its danger is Colin McGinn (popularizer of mysterianism) in his The Making of a Philosopher. He says that what attracted him to philosophy was the ability to reason ones way to contrarian opinions. Being forced to an answer itself has an appeal. This is a major problem in the transhumanist and libertarian communities, for example, where bullet biting is much more highly regarded than having your facts straight.

mtraven,

Got a better one?

Biology and physics. Google Tim Van Gelder for a philosophical perspective on the benefits of using dynamics to explain cognition. I think he has papers online.

Presumably your brain is processing symbols right now, as your read this.

I think there's an important distinction between being able to manipulate symbols and engaging in symbol processing. After all, I can use a hammer, but nobody thinks there's hammers in my brain.

Caledonian,

But computer programmers don't need to understand the hardware, either. Do you think they c

Eliezer, serious question, why don't you re-brand your project as designing an Self-Improving Automated Science Machine rather than a Seed AI and generalize Friendly AI to Friendly Optimization (or similar)? It seems to me that: (a) this would be more accurate since it's not obvious (to me at least) that individual humans straightforwardly exhibit the traits you describe as "intelligence"; and (b) you'd avoid 90% of the criticism directed at you. You could, for example, avoid the usual "people have been promising AI for 60 years" line of argument.

mtraven, The computer started as an attempt to mechanize calculation. There's a tradition in mathematics, going back to the Greeks and popular with mathematicians, that mathematics is exemplary reasoning. It's likely that identifying computation and thought builds off that. If calculation/mathematics is exemplary thought and computers mechanize calculation then computers mechanize thought.

I would argue instead that mathematics is actually exemplary (albeit creative) tool-use. This is especially stark if you look at the original human computers Caledonian m... (read more)

mtraven, I think your example demonstrates well why computationalism rests on a basic error. The type-token relationship between A-ness and instances of the letter "A" is easily explained: what constitutes A-ness is a social convention and the various diverse instances of "A" are produced as human artifacts with reference to that convention. They all exhibit A-ness because we made them that way. Computers are like this too. Computers can be made from different substrates because they only have to conform to our conventions of how a comp... (read more)

Eliezer, you're spot on with the "Determinator." The modern free will debate has its roots not in the clockwork universe of Newtonianism but the supposed problem of God's omnipotence and omniscience. The problem of free will was originally formulated in terms of a Determinator - God - who chose and imminently caused the future. The question was "How can we also have free will?" and free will was, of course, also an important concept in Christian theology (we're made in God's image and therefore chose and cause our futures just like God ... (read more)

Crush on Lyle,

But "if your actions are determined by prior causes" then whether or not you think those actions are blameworthy is determined by prior causes too. The act of punishing criminals is subject to the same physics that crime is. So is talking about the act of punishing criminals. And so on.

I agree. But no philosopher is going to bite that bullet. They'd be out of a job.

The "Why punish criminals?" question has a long history. The idea is that if your actions are determined by prior causes then you're no longer blameworthy. I think for most people deterrence would be morally unacceptable if they did not also consider criminals blameworthy. Why not punish their friends and families if that would also act as an effective deterrent? Actually this question - how can we delimit external and internal causes - is more interesting to me than general concepts of free will (short answer: we can't). If you want a nice examp... (read more)

"Free will" is one of those concepts in philosophy where I have absolutely no idea what it's supposed to be about. I've read a few works on the subject and they all assure me that everyone is convinced they have it. I think the lesson to be learned is that words and concepts have histories of their own and frequently fall out of touch with reality completely. I think "free will" is like that.

Lots of physicists don't believe in many-worlds because they believe in some other theory or interpretation. Parsimony is often used to dismiss many-worlds; mainly because many-worlds doesn't make any predictions so it's difficult to refute on other grounds. That doesn't make it true of course. If you have reason to believe that some other theory or interpretation is worth pursuing then you probably won't spend much time refuting many-worlds. So parsimony will be the lazy way to dismiss many-worlds but not the reason you hold another view.

The reason most p... (read more)

I knew this was where we were headed when you started talking about zombies and I knew exactly what the error would be.

Even if I accept your premises of many-worlds and timeless physics, the identity argument still has exactly the same form as it did before. Most people are aware that atomic-level identity is problematic even if they're not aware of the implications of quantum physics. They know this because they consume and excrete material. Nobody who's thought about this for more than a few seconds thinks their identity lies in the identity of the atoms... (read more)

Eliezer: As I said, there are plenty of circular definitions of intelligence, such as defining it as an "powerful optimization process" that hones in on outcomes you've predefined as being the product of intelligence (which is what your KnowabilityOfAI appears to do). Perhaps for your needs such a (circular) operational definition would suffice: take the set of artifacts and work backwards. That hardly seems helpful in designing any sort of workable software system though.

Re: modeling the human brain. Modeling the human brain would involve higher... (read more)