I've watched Stuart Russel's TED talk on AI risk, and my gut reaction to it was "do you want to be paperclips? this is how you become paperclips!". It goes completely against the grain of the view that has been expressed on this blog as of few years ago. But, then again, AI is hard, and there might be some recent developments that I have missed. What is the current state of the research? What does EY and his camarilla think about the state of the problem as of now?
Confirm.
It's not about qualia. It's about any arbitrary property.
Imagine a cookie like Oreo to the last atom, except that it's deadly poisonous, weighs 100 tons and runs away when scared.
Chalmers does not claim that p-zombies are logically possible, he claims that they are metaphysically possible. Chalmers already believes that certain atomic configurations necessarily imply consciousness, by dint of psychophysical laws.
Okay. In that case, I peg his argument as proving too much. Imagine a cookie that is exactly like an Oreo, down to the last atom, except it's raspberry flavored. This situation is semantically the same as a p-Zombie, so it's exactly as metaphysically possible, whatever that means. Does it prove that raspberry flavor is an extra, nonphysical fact about cookies?
This argument is not going to win over their heads and hearts. It's clearly written for a reductionist reader, who accepts concepts such as Occam's Razor and knowing-what-a-correct-theory-looks-like. But such a person would not have any problems with p-Zombies to begin with.
If you want to persuade someone who's been persuaded by Chalmers, you should debunk the argument itself, not bring it to your own epistemological ground where the argument is obviously absurd. Because you, and the Chalmers-supporter are not on the same epistemological ground, and will probably never be.
Here's how you would do that.
---- START ARGUMENT ----
Is it conceivable that the 5789312365453423234th digit of Pi is 7?
No, don't look it up just yet. Is it conceivable to you, right now, that it's 7?
For me, yes, it is. If I look it up, and it turns out to be 7, I would not be surprised at all. It's a perfectly reasonable outcome, with predictable consequences. It's not that hard for me to imagine me running a program that calculates and prints the number, and it printing out 7.
Yet, until you look it up, you don't really know if it's 7 or not. It could be 5. It would also be a reasonable, non-surprising and conceivable outcome.
Yet at least one of those outcomes is logically impossible. The exact value of Pi is logically determined, and, if you believe that purely logical conclusions apply universally, then one of those values of 5789312365453423234th digit of Pi is universally impossible.
And yet both are conceivable.
So logical impossibility does not imply inconceivability. This is logically equivalent to saying "conceivability does not imply logical possibility" (A->B => ~B->~A).
If conceivability does not imply logical possibility, then even if you can imagine a Zombie world, it does not mean that the Zombie world is logically possible. It may be the case that the Zombie world is logically impossible. Chalmer's argument does not rule that out. For example, it may be the case that certain atomic configurations necessarily imply consciousness. Or it may be any other case of logical impossibility. What matters is that consciousness as an additional nonphysical entity is not implied by its conceivability.
---- END ARGUMENT ----
I have taken the survey.
Sort of yes. Maybe not sufficiently new. I shall look into it.
I am definitely not better off without what I lost. Genuine curiosity had tremendously powerful effect on my learning.
The source of my wanting is conscience rather than passion, though. It's a completely different thing, and learning is a tiring activity which importance I realize, rather than something that empowers me or something I look forward to. That's the problem.
What's wrong with ethanol made from corn, anyway?