All of Marcus_Oz's Comments + Replies

I really enjoyed reading this. Quite concise, well organised and I thought quite comprehensive (nothing is ever exhaustive so no need to apologise on that front). I will find this a very useful resource and while nothing in it was completely "new" to me I found the structure really helped me to think more clearly about this. So thanks.

A suggestion - might be useful to turn your attention more to specific process steps using the attention directing classification tools outlined here. For example

Step 1: Identify type of risk (transparent, Opaque, K... (read more)

I have not been to one of these before. I think I should be able to get there depending on my daughter's work schedule. Is it okay just to turn up? :)

Whilst I appreciate the validity of criticism offered here of the use of the word emergence (by itself) as if were an explanation sufficient unto itself - I think it a little harsh. To call it "futile" is almost acting as semantic stop sign itself for the term.

We need to take a little time to properly understand what is meant by emergence when used properly.

First that it is an observation rather than an explnation. But an observation with useful descriptive power since it observes that the phenomena under consideration is a process with properti... (read more)

Hi Capla - no that is not what Godel's theorem says (actually there are two incompleteness theorems)

1) Godel's theorems don't talk about what is knowable - only about what is (formally) provable in a mathematical or logic sense

2) The first incompleteness theorem states that no consistent system of axioms whose theorems can be listed by an any sort of algorithm is capable of proving all truths about the relations of the natural numbers. In other words for any such system, there will always be statements about the natural numbers that are true, but that are ... (read more)

-1Capla
Thank you, A little bit more informed. My issue with consciousness involves p-zombies. Any experiment that wanted to understand consciousness, would have to be able to detect it, which seems to me to be philosophically impossible. To be more specific, any scientific investigation of the cause of consciousness would have (to simplify) an independent variable that we could manipulate to see if consciousness is present or not, depending on the manipulated variable. We assume that those around us are conscious, and we have good reason to do so, but we can't rely on that assumption in any experiment in which we are investigating consciousness. As Eliezer points out, that an individual says he's conscious is a pretty good signal of consciousness, but we can't necessarily rely on that signal for non-human minds. A conscious AI may never talk about it's internal states depending on its structure (humans have a survival advantage to social sharing of internal realities). On the flip side, a savvy but non-conscious AI, may talk about it's "internal states" because it is guessing the teacher's password in the realist way imaginable: it has no understanding whatsoever of what those state are, but computes that aping them will accomplish it's goals. I don't know how we could possibly know if the AI is aping conciseness for it own ends or if it actually is conscious. If consciousness is thus undetectable, I can't see how science can investigate it. That said, I am very well aware that “Throughout history, every mystery, ever solved has turned out to be not magic” and that ever single time something has seemed inscrutable to science, a reductionist explanation eventually, surfaced. Knowing this, I have to seriously down grade my confidence that "No, really, this time it is different. Science really can't pierce this veil." I look forward to someone coming forward with somthign clever that dissolves the question, but even so, it does seem inscrutable.