Well, his comment was deleted, possibly by him, so we should take that into account - maybe he thought he was being a bit overly Cassandra-like too.

The other thing to remember is that Musk's comments reach a slightly different audience to the usual with regards to AI risk. So it's at least somewhat relevant to see the perspective of the one communicating to these people.

What did you think of Bostrom's recent book?

By "we" do you mean Gök Us Sibernetik Ar & Ge in Turkey? How many people work there?

We believe we can achieve trans-sapient performance by 2018

What does "trans-sapient performance" mean?

We believe we can achieve trans-sapient performance by 2018, he is not that off the mark. But dangers as such, those are highly over-blown, exaggerated, pseudo-scientific fears, as always.

Phrasing "human-level" as "roughly as good as humans at science etc." (in the questions) is incorrect, because it requires the AI to be human-like in their ability. Instead, it should be something like "roughly as good as humans (or better, perhaps unevenly) at science etc.". That parenthetical is important, as it distinguishes the magical matching of multiple abilities to human level, which respondents rightly object to, from a more well-defined lower bound where you require that it's at least as capable.

Thanks for the nice interview Alexander. I'm Eray Ozkural by the way, if you have any further questions, I would love to answer them.

I actually think that SIAI is serving a useful purpose when they are highlighting the importance of ethics in AI research and computer science in general.

Most engineers, whether because we are slightly autistic or not I do not know, have no or very little interest in ethical consequences of their work. I have met many good engineers who work for military firms (firms that thrive on easy money from the military), and not once have they raised a point about ethics. Neither do data mining or information retrieval researchers seem to have such qualms (except when they are pretending to, at academic conferences). At companies like facebook, they think they have a right to exploit the data they have collected and use it for all sorts of commercial and police state purposes. Likewise in AI and robotics, I see people cheering whenever military drones or robots are mentioned, as if automatization of warfare is civil or better in some sense because it is higher technology.

I think that at least AGI researchers must understand that they must have no dealing with the military and the government, and by doing so, they may be putting themselves and all of us at risk. Maybe fear tactics will work, I don't know.

On the other hand, I don't think that "friendly" AI is such a big concern, for reasons I mention above, artificial persons simply aren't needed. I have heard the argument that "but someone will build it sooner or later", though there is no reason that person is going to listen to you. The way I see it, it's better to focus on technology right now, so we can have a better sense of applications first. People seem to think that we should equip robots with fully autonomous AGI. Why is that? People have mentioned to me robotic bartenders, robotic geishas, fire rescuers, and cleaners. Well, is that serious? Do you really want a bartender that can solve general relativity problems while cleaning glasses? It's just nonsense. Or does a fire rescuer really need to think about whether it wants to go on to exterminate the human race after extinguishing the fire? The simple answer is that the people who give those examples are not focusing on the engineering requirements of the applications they have in mind. Another example: military robots. They think that military robots must have a sense of morality. I ask you, why is it important to have moral individuals in an enterprise that is fundamentally immoral? All war is murder, and I suggest you to stay away from professional murder business. That is, if you have any true sense of morality.

Instead of "friendly", a sense of "benevolence" may instead be thought, and that might make sense from an ethical theory viewpoint. It is possible to formalize some theories of ethics and implement them on an autonomous AI, however, for all the capabilities that autonomous trans-sapient AI's may possess, I think it is not a good idea to let such machines develop into distinctive personalities of their own, or meddle in human affairs. I think there are already too many people on earth, I don't think we need artificial persons. We might need robots, we might need AI's, but not artificial persons, or AI's that will decide instead of us. I prefer that as humans we remain at the helm. That I say with respect to some totalitarian sounding proposals like CEV. In general, I do not think that we need to replace critical decision making with AI's. Give AI's to us scientists and engineers and that shall be enough. For the rest, like replacing corrupt and ineffective politicians, a broken economic system, social injustice, etc., we need human solutions, because ultimately we must replace some sub-standard human models with harmful motivations like greed and superstitious ideas, with better human models that have the intellectual capacity to understand the human condition, science, philosophy, etc., regardless of any progress in AI. :) In the end, there is a pandemic of stupidity and ignorance that we must cure for those social problems, and I doubt we can cure it with an AI vaccine.

There are in fact some plausible scientific hypotheses that try to isolate particular physical states associated with "qualia". Without giving references to those, obviously, as I'm sure you'll all agree, there is no reason to debate the truth of physicalism.

I was looking some things up after you mentioned this, and after reading a bit about it, qualia appears to be extremely similar to sensory memory.

(http://en.wikipedia.org/wiki/Qualia) (http://en.wikipedia.org/wiki/Sensory_memory)

These quotes about them from Wikipedia(with the names removed) seem to do a good job describing the similarity:

'The information represented in ### is the "raw data" which provides a snapshot of a person's overall sensory experience.' 'Another way of defining ### is as "raw feels." A raw feel is a perception in and of itself, considered entirely in isolation from any effect it might have on behavior and behavioral disposition.'

If you think about this in P-zombie terms, and someone attempts to say "A P-zombie is a person who has sensory memory, but not qualia." I'm not sure what would even be the difference between that and a regular person. Either one can call on their sensory memory to say "I am experiencing redness right now, and now I am experiencing my experiences of redness" and it would seem to be correct if that is what is in their sensory memory. There doesn't appear to be anything left for qualia to explain, and it feels a lot like the question is dissolved at that point.

Is this approximately correct, or is there something else that qualia attempts to explain that sensory memory doesn't that I'm not perceiving?

There are in fact some plausible scientific hypotheses that try to isolate particular physical states associated with "qualia". Without giving references to those, obviously, as I'm sure you'll all agree, there is no reason to debate the truth of physicalism.

The mentioned approach is probably bogus, and seems to be a rip-off of Marvin Minsky's older A-B brain ideas in "The Society of Mind". I wish I were a "cognitive scientist" it would be so much easier to publish!

However, needless to say any such hypothesis must be founded on the correct philosophical explanation, which is pretty much neurophysiological identity theory. I don't see a need to debate that, either. Debates of dualism etc. are for the weak minded.

Furthermore, awareness is not quite the same thing as phenomenal consciousness, either. Awareness itself is a quite high cognitive function. But a system could have phenomenal consciousness without any discernible perceptual awareness. I suspect that these theories are not sufficiently informed by neuroscience and philosophy, but neither am I going to offer free clues about the solution to that :) For now, let us just say that it is entirely plausible that small nervous systems (like that of an insect) with no possibility of higher order representations still may have subjective experience. There is also a hint of anthropocentricism in the cited approach (we're conscious because we can make those higher order representations...), which I usually think points to the falsehood of a theory of mind (similar errors are often seen on this site, as well).

Is Dennett to blame here? I hope not :/ Dennett has many excellent ideas, but his approach to consciousness may push the people the wrong way (as it has some flavor of behaviorism, which is not the most advanced view).