TAG

Scientist by training, coder by previous session,philosopher by inclination, musician against public demand.

https://theancientgeek.substack.com/?utm_source=substack&utm_medium=web&utm_campaign=substack_profile

Posts

Sorted by New

Wiki Contributions

Comments

Sorted by
TAG-1-5

"it" isn't a single theory.

The argument that Everettian MW is favoured by Solomonoff induction, is flawed.

If the program running the SWE outputs information about all worlds on a single output tape, they are going to have to be concatenated or interleaved somehow. Which means that to make use of the information, you gave to identify the subset of bits relating to your world. That's extra complexity which isn't accounted for because it's being done by hand, as it were..

TAG20

By far the best definition I’ve ever heard of the supernatural is Richard Carrier’s: A “supernatural” explanation appeals to ontologically basic mental things, mental entities that cannot be reduced to nonmental entities.

Physicalism, materialism, empiricism, and reductionism are clearly similar ideas, but not identical. Carrier's criterion captures something about a supernatural ontology, but nothing about supernatural epistemology. Surely the central claim of natural epistemology is that you have to look...you can't rely on faith , or clear ideas implanted in our minds by God.

it seems that we have very good grounds for excluding supernatural explanations a priori

But making reductionism aprioristic arguably makes it less scientific...at least, what you gain in scientific ontology, you lose in scientific epistemology.

I mean, what would the universe look like if reductionism were false

We wouldn't have reductive explanations of some apparently high level phenomena ... Which we don't.

I previously defined the reductionist thesis as follows: human minds create multi-level models of reality in which high-level patterns and low-level patterns are separately and explicitly represented. A physicist knows Newton’s equation for gravity, Einstein’s equation for gravity, and the derivation of the former as a low-speed approximation of the latter. But these three separate mental representations, are only a convenience of human cognition. It is not that reality itself has an Einstein equation that governs at high speeds, a Newton equation that governs at low speeds, and a “bridging law” that smooths the interface. Reality itself has only a single level, Einsteinian gravity. It is only the Mind Projection Fallacy that makes some people talk as if the higher levels could have a separate existence—different levels of organization can have separate representations in human maps, but the territory itself is a single unified low-level mathematical object. Suppose this were wrong.

Suppose that the Mind Projection Fallacy was not a fallacy, but simply true.

Note that there are four possibilities here...

  1. I assume a one level universe, all further details are correct.

  2. I assume a one level universe, some details may be incorrect

  3. I assume a multi level universe, all further details are correct.

  4. I assume a multi level universe, some details may be incorrect.

How do we know that the MPF is actually fallacious, and what does it mean anyway?

If all forms of mind projection projection are wrong, then reductive physicalism is wrong, because quarks, or whatever is ultimately real, should not be mind projected, either.

If no higher level concept should be mind projected, then reducible higher level concepts shouldn't be ...which is not EY's intention.

Well, maybe irreducible high level concepts are the ones that shouldn't be mind projected.

That certainly amounts to disbelieving in non reductionism...but it doesn't have much to do with mind projection. If some examples of mind projection are acceptable , and the unacceptable ones coincide with the ones forbidden by reductivism, then MPF is being used as a Trojan horse for reductionism.

And if reductionism is an obvious truth , it could have stood on its own as apriori truth.

Suppose that a 747 had a fundamental physical existence apart from the quarks making up the 747. What experimental observations would you expect to make, if you found yourself in such a universe?

Science isn't 100% observation,it's a mixture of observation and explanation.

A reductionist ontology is a one level universe: the evidence for it is the success of reductive explanation , the ability to explain higher level phenomena entirely in terms of lower level behaviour. And the existence of explanations is aposteriori, without being observational data, in the usual sense. Explanations are abductive,not inductive or deductive.

As before, you should expect to be able to make reductive explanations of all high level phenomena in a one level universe....if you are sufficiently intelligent. It's like the Laplace's Demon illustration of determinism,only "vertical". If you find yourself unable to make reductive explanations of all phenomena, that might be because you lack the intelligence , or because you are in a non reductive multi level universe or because you haven't had enough time...

Either way, it's doubtful and aposteriori, not certain and apriori.

If you can’t come up with a good answer to that, it’s not observation that’s ruling out “non-reductionist” beliefs, but a priori logical incoherence"

I think I have answered that. I don't need observations to rule it out. Observations-rule it-in, and incoherence-rules-it-out aren't the only options.

People who live in reductionist universes cannot concretely envision non-reductionist universes.

Which is a funny thing to say, since science was non-reductionist till about 100 years ago.

One of the clinching arguments for reductionism.was the Schrödinger equation, which showed that in principle, the whole of chemistry is reducible to physics, while the rise of milecular biology showeds th rreducxibility of Before that, educators would point to the de facto hierarchy of the sciences -- physics, chemistry, biology, psychology, sociology -- as evidence of a multi-layer reality.

Unless the point is about "concretely". What does it mean to concretely envision a reductionist universe? Pehaps it means you imagine all the prima facie layers, and also reductive explanations linking them. But then the non-reductionist universe would require less envisioning, because byit's the same thing without the bridging explanations! Or maybe it means just envisioing huge arrays of quarks. Which you can't do. The reductionist world view , in combination with the limitations of the brain, implies that you pretty much have to use higher level, summarised concepts...and that they are not necessarily wrong.

But now we get to the dilemma: if the staid conventional normal boring understanding of physics and the brain is correct, there’s no way in principle that a human being can concretely envision, and derive testable experimental predictions about, an alternate universe in which things are irreducibly mental. Because, if the boring old normal model is correct, your brain is made of quarks, and so your brain will only be able to envision and concretely predict things that can predicted by quarks.

  1. "Your brain is made of quarks" is aposteriori, not apriori.

  2. Your brain being made of quarks doesn't imply anything about computability. In fact, the computatbolity of the ultimately correct version of quantum physics is an open question.

  3. Incomputability isn't the only thing that implies irreducibility, as @ChronoDas points out.

  4. Non reductionism is conceivable, or there would be no need to argue for reductionism.

TAG31

But focussing on the weight isn't obviously correct , ethically. You cant assume that the answer to "what do I expect to see" will work the same as the answer to "what should I do". Is-ought gap and all that.

Its tempting to think that you can apply a standard decision theory in terms of expected value to Many Worlds, since it is a matter of multiplying subjective value by probability. It seems reasonable to assess the moral weight of someone else's experiences and existence from their point of view. (Edit: also, our experiences seem fully real to us, although we are unlikely to be in a high measure world) That is the intuition behind the common rationalist/utilitarian/EA view that human lives don't decline in moral worth with distance. So why should they decline with lower quantum mechanical measure?

There is quandary here: sticking to the usual "adds up to normality" principle,as an apriori axiom means discounting the ethical importance of low-measure worlds in order to keep your favourite decision theory operating in the usual single-universe way...even if you are in a multiverse. But sticking to the equally usual universalist axiom, that you dont get to discount someone's moral worth on the basis of factors that aren't intrinsic to them, means you should not discount..and that the usual decision theory does not apply.

Basically, there is a tension between four things Rationalists are inclined to believe in:-

  • Some kind of MWI is true.

  • Some utilitarian and universalist ethics is true.

  • Subjective things like suffering are ethically relevant. It's not all about number of kittens

  • It's all business as normal...orchards up to normality.. fundamental ontological differences should not affect your decision theory.

TAG-1-3

According the many-worlds interpretation (MWI) of quantum mechanics, the universe is constantly splitting into a staggeringly large number of decoherent branches containing galaxies, civilizations, and people exactly like you and me

There is more than one many worlds interpretation. The version stated above is not known to be true.

There is an approach to MWI based on coherent superpositions, and a version based on decoherence. These are (for all practical purposes) incompatible. Coherent splitting gives you the very large numbers of "worlds"..except that they are not worlds, conceptually.

Many worlders are pointing at something in the physics and saying "that's a world"....but whether it qualifies as a world is a separate question , and a separate kind of question, from whether it is really there in the physics. One would expect a world, or universe, to be large, stable, non-interacting, objective and so on . A successful MWI needs to jump three hurdles: mathematical correctness, conceptual correctness, and empirical correctness.

Decoherent branches are expected to be large, stable, non interacting, objective and irreversible...everything that would be intuitively expected of a "world". But there is no empirical evidence for them , nor are they obviously supported by the core mathematics of quantum mechanics, the Schrödinger equation. Coherent superpositions are small scale , down to single particles, observer dependent, reversible, and continue to interact (strictly speaking , interfere) after "splitting".

(Note that Wallace has given up on the objectivity of decoherent branches. That's another indication that MWI is not a single theory).

There isn’t the slightest evidence that irrevocable splitting, splitting into decoherent branches occurs at every microscopic event -- that would be combining the frequency of coherent style splitting with the finality of decoherent splitting. We dont know much about decoherence , but we know it is a multi-particle process that takes time, so decoherent splitting, if there is such a thing, must be rarer than the frequency of single particle interactions. ( And so decoherence isn't simple ). As well as the conceptual incoherence, there is In fact plenty of evidence—eg. the existence of quantum computing—that it doesnt work that way

Also see

https://www.lesswrong.com/posts/wvGqjZEZoYnsS5xfn/any-evidence-or-reason-to-expect-a-multiverse-everett?commentId=o6RzrFRCiE5kr3xD4

I’m not going to argue for this view as that was done very well by Eliezer in his Quantum Physics.

Which view? Coherent many worlds? Decoherent many worlds? Wallace's view?

@dr_s

I feel like branches being in fact an uncountable continuum is essentially a given

Decohereht branches being being countable, uncountable, or anything else is not given, since there is no established theory of of decoherence.

I mean that the amount of universes that is created will be created anyway, just as a consequence of time passing. So it doesn’t matter anyway. If your actions e.g. cause misery in 20% of those worlds, then the fraction is all that matters; the worlds will exist anyway, and the total amount is not something you’re affecting or controlling.

That's a special case of "no moral responsibility under determinism". which might be true , but it's very different from "utilitarianism works fine under MWI".

Enough of the physics confusions -- onto the ethics confusions!

As well as confusion over the correct version of many worlds, there is of course confusion about which theory of ethics is correct.

There's broadly three areas where MWI has ethical implications. One is concerned with determinism, freedom of choice, and moral responsibility. One is over the fact that MW means low probability events have to happen every time -- as opposed to single universe physics, where they usually don't. The other is over whether they are discounted in moral significance for being low in quantum mechanical measure or probability

MWI and Free Will

MWI allows probabilities of world states to change over time, but doesn't allow them to be changed, in a sense amounting to libertarian free will. Agents are just part of the universal wave function, not anything outside the system, or operating by different rules.MWI is, as it's proponents claim, a deterministic theory, and it only differs from single world determinism in that possible actions can't be refrained from, and possible futures can't be avoided. Alternative possibilities are realities, in other words.

MWI, Moral Responsibility, and Refraining.

A standard argument holds that causal determinism excludes libertarian free will by removing alternative possibilities. Without alternative possibilities, you could but have done other than you did, and , the argument goes, you cannot be held responsible for what you had no choice but to do.

Many world strongly implies that you make all possible decisions: according to David Deutsch's argument that means it allows alternative possibilities, and so removes the objection from moral responsibility despite being a basically deterministic theory.

However, deontology assumed that performing a required act involves restraining from alternatives.. and that it is possible to retain from forbidden acts. Neither is possible under many worlds. Many worlds creates the possibility, indeed the necessity, of doing otherwise, but removes the possibility of refraining from an act. Even though many worlds allows Alternative Possibilities, unfortunately for Deutschs argument, that other aspects create a similar objection on the basis of moral responsibility: why would you hold someone morally responsible for an act if they could not refrain from it?

MWI, Probability, and Utilitarian Ethics

Its tempting to think that you can apply a standard decision theory in terms of expected value to Many Worlds, since it is a matter of multiplying subjective value by probability. One wrinkle is that QM measure isn't probability -- the probability of something occurring or not -- because all possible branches occur in MWI. Another is that it is reasonable to assess the moral weight of someone else's experiences and existence from their point of view. That is the intuition behind the common rationalist/utilitarian/EA view that human lives don't decline in moral worth with distance. So why should they decline with lower quantum mechanical measure? There is quandary here: sticking to the usual "adds up to normality" principle,as an apriori axiom means discounting the ethical importance of low-measure worlds in order to keep your favourite decision theory operating in the usual single-universe way, even if you are in a multiverse. But sticking to the equally usual universalist axiom, that that you dont get to discount someone's moral worth on the basis of factors that aren't intrinsic to them, means you should not

Measure is not probability.

Mathematically, Quantum mechanical measure—amplitude—isn’t ordinary probability, which is why you need the Born rule.The point of the Born rule is to get a set of ordinary probabilities, which you can then test frequentistically, over a run of experiments. Ontologcally, it also not probability, because it does not represent the likelihood of one happening instead of another. And it has its own role, unlike that if ordinary probability, which is explaining how much contribution to a coherent superposition each component state makes (although what that means in the case of irrevocably decohered branches is unclear)

Whether you are supposed to care about them ethically is very unclear, since it is not clear how utilitarian style ethics would apply, even if you could make sense of the probabilities. But you are not supposed to care about them for the purposes of doing science, since they can no longer make any difference to your branch. MWI works like a collapse theory in practice.

MWI creates the puzzle that low probability outcomes still happen, and have to be taken into account ethically. Many rationalists assume that they simply matter less, because that is the only way to restore anything like a normal view of ethical action -- but one should.not assume something merely because it is convenient.

It can be argued that most probability calculations come out the same under different interpretations of QM...but ethics is different. It is not clear how low measure worlds should be considered in utilitarian ethics. It's tempting to ethically discount low measure worlds in some way, because that most closely approximates conventional single world utilitarianism. The alternative might force one to the conclusion that overall good outcomes are imposed to attain , so long as one cannot reduced the measures of worlds full of suffering zero. However, one should not jump to the conclusion that something is true just because it is convenient. And of course, MWI is a scientific theory so it doesn't comes with built in ethics

One part of the problem is that QM measure isn't probability, because all possible branches occur in MWI. Another stems from the fact that what other people experience is relevant to them, wheareas for a probability calculation, I only need to be able to statistically predict my own observations. Using QM to predict my own observations, I can ignore the question of whether something has a ten percent chance of happening in the one and only world, or a certainty of happening in one tenth of possible worlds. However, these are not necessarily.equivalent ethically.

Suppose they low measure worlds are discounted ethically. If people in low measure worlds experience their suffering fully, then a 1%, of creating a hell-world would be equivalent in suffering to a 100% chance, and discount is unjustified. But if people in low measure worlds are like philosophical zombies, with little or no phenomenal consciousness, so that their sensations are faint or nonexistent, the moral hazard is much lower, and the discount is justified. A point against discounting is that *our* experiences seem fully real to *us*, although we are unlikely to be in a high measure world

A similar, but slightly less obvious argument applies to causing death. Causing the "death" of a complete zombie is presumably as morally culpable as causing the death of a character in a video game...which, by common consent, is not problem at all. So... causing the death of a 50% zombie would be only half as bad as killing a real person...maybe.

Classical Measure isn't Quantum Mechanical Measure

A large classical universe is analogous to Many Worlds in that the same structures -- the same people and planets -- repeat over long distances. It's even possible to define a measure, by counting repetitions up to a certain level of similarity. It as also possible to think about Quantum Mechanical measure way, but it is not following the maths -- there's nothing in the formalism that looks like summing a number of identical worlds to get a measure. So, again, it’s an extraneous assumption. .

Of course, MWI doesn't directly answer the question about consciousness and zombiehood .You can have objective information about observations, and if your probability calculus is wrong , you will get wrong results and know that you are getting wrong results. That is the negative feedback that allows physics to be less wrong. And you can have subjective information about your own mental states, and if your personal calculus is wrong , you will get wrong results and know that you are getting wrong results. That is the negative feedback that allows personal decision theory to be less wrong.

Altruistic ethics is different. You don't have either kind of direct evidence, because you are concerned with other people's subjective sensations , not objective evidence , or your own subjectivity. Questions about ethics are downstream of questions about qualia, and qualia are subjective, and because they are subjective, there is no reason to expect them to behave like third person observations.

"But it all adds up to normality!"

If "it all" means every conjecture you can come up with, no It doesn't. Most conjectures are wrong. The point of empirical testing is to pick out the right ones -- if you write down some random theory, it probably doesnt up to reality, sincemost theories are wrong. That's a difficult process, not something you get for free. So "it all adds up to normality" is not some universal truth And ethical theories relating to come be else's feelings are difficult to test, especially if someone else is in the far future, or an unobservable branch of the multiverse. Testability isn't an automatic given either.

TAG20

Every quantum event splits the multiverse, so my measure should decline by 20 orders of magnitude every second.

There isn’t the slightest evidence that irrevocable splitting, splitting into decoherent branches occurs at every microscopic event -- that would be combining the frequency of coherentism style splitting with the finality of decoherent splitting. As well as the conceptual incoherence, there is In fact plenty of evidence—eg. the existence of quantum computing—that it doesnt work that way

"David Deutsch, one of the founders of quantum computing in the 1980s, certainly thinks that it would. Though to be fair, Deutsch thinks the impact would “merely” be psychological – since for him, quantum mechanics has already proved the existence of parallel uni- verses! Deutsch is fond of asking questions like the following: if Shor’s algorithm succeeds in factoring a 3000-digit integer, then where was the number factored? Where did the computational resources needed to factor the number come from, if not from some sort of “multiverse” exponentially bigger than the universe we see? To my mind, Deutsch seems to be tacitly assuming here that factoring is not in BPP – but no matter; for purposes of argument, we can certainly grant him that assumption. It should surprise no one that Deutsch’s views about this are far from universally accepted. Many who agree about the possibil- ity of building quantum computers, and the formalism needed to describe them, nevertheless disagree that the formalism is best inter- preted in terms of “parallel universes.” To Deutsch, these people are simply intellectual wusses – like the churchmen who agreed that the Copernican system was practically useful, so long as one remembers that obviously the Earth doesn’t really go around the sun. So, how do the intellectual wusses respond to the charges? For one thing, they point out that viewing a quantum computer in terms of “parallel universes” raises serious difficulties of its own. In particular, there’s what those condemned to worry about such things call the “preferred basis problem.” The problem is basically this: how do we define a “split” between one parallel universe and another? There are infinitely many ways you could imagine slic- ing up a quantum state, and it’s not clear why one is better than another! One can push the argument further. The key thing that quan- tum computers rely on for speedups – indeed, the thing that makes quantum mechanics different from classical probability theory in the first place – is interference between positive and negative amplitudes. But to whatever extent different “branches” of the multiverse can usefully interfere for quantum computing, to that extent they don’t seem like separate branches at all! I mean, the whole point of inter- ference is to mix branches together so that they lose their individual identities. If they retain their identities, then for exactly that reason we don’t see interference. Of course, a many-worlder could respond that, in order to lose their separate identities by interfering with each other, the branches had to be there in the first place! And the argument could go on (indeed, has gone on) for quite a while. Rather than take sides in this fraught, fascinating, but perhaps ultimately meaningless debate..."..Scott Aaronson , QCSD, p148

Also see

https://www.lesswrong.com/posts/wvGqjZEZoYnsS5xfn/any-evidence-or-reason-to-expect-a-multiverse-everett?commentId=o6RzrFRCiE5kr3xD4

TAG4-2

It seems common for people trying to talk about AI extinction to get hung up on whether statements derived from abstract theories containing mentalistic atoms can have objective truth or falsity values. They can. And if we can first agree on such basic elements of our ontology/epistemology as that one agent can be objectively smarter than another, that we can know whether something that lives in a physical substrate that is unlike ours is conscious, and that there can be some degree of objective truth as to what is valuable [not that all beings that are merely intelligent will necessarily pursue these things], it in fact becomes much more natural to make clear statements and judgments in the abstract or general case, about what very smart non-aligned agents will in fact do to the physical world.

Why does any of that matter for AI safety? AI safety is a matter of public policy. In public policy making, you have a set of preferences, which you get from votes or surveys, and you formulate policy based on your best objective understanding of cause and effect. The preferences don't have to be objective, because they are taken as given. It's quite different to philosophy, because you are trying to achieve or avoid something, not figure out what something ultimately is. You do t have to answer Wolfram's questions in their own terms, because you can challenge the framing.

And if we can first agree on such basic elements of our ontology/epistemology as that one agent can be objectively smarter than another,

It's not all that relevant to AI safety, because an AI only needs some potentially dangerous capabilities. Admittedly, a lot of the literature gives the opposite impression.

that we can know whether something that lives in a physical substrate that is unlike ours is conscious,

You haven't defined consciousness and you haven't explained how . It doesn't follow automatically from considerations about intelligence. And it doesn't follow from having some mentalistic terms in our theories.

and that there can be some degree of objective truth as to what is valuable

there doesn't need to be. You don't have to solve ethics to set policy.

TAG20

Arguably, “basic logical principles” are those that are true in natural language.

That's where the problem starts, not where it stops. Natural language supports a bunch of assumptions that are hard to formally reconcile: if you want your strict PNC, you have to give up on something else. The whole 2500 yeah history of logic has been a history of trying to come up with formal systems that fulfil various desiderata. It is now formally proven that you can't have all of them at once, and it's not obvious what to keep and what to ditch. (Godelian problems can be avoided with lower power systems, but that's another tradeoff, since high power is desirable).

Formalists are happy to pick a system that's appropriate for a practical domain, and to explore the theoretical properties of different systems in parallel.

Platonists believe that only one axiom system has truth in addition to usefulness, but can't agree which one it is, so it makes no difference in practice

I'm not seeing a specific problem with sets -- you can avoid some of the problems of naive self theory by adding limitations, but that's tradeoffs again.

Otherwise nothing stops us from considering absurd logical systems where “true and true” is false, or the like.

"You can't have all the intuitive principles in full strength in one system"

doesn't imply

"adopt unintuitive axioms".

Even formalists don't believe all axiomisations are equally useful.

Likewise, “one plus one is two” seems to be a “basic mathematical principle” in natural language.

What's 12+1?

Any axiomatization which produces “one plus one is three” can be dismissed on grounds of contradicting the meanings of terms like “one” or “plus” in natural language.

They're ambiguous in natural language, hence the need for formalisation.

The trouble with set theory is that, unlike logic or arithmetic, it often doesn’t involve strong intuitions from natural language.

It involves some intuitions . It works like clubs. Being a senator is being a member of a set, not exemplifying a universal.

Sets are a fairly artificial concept compared to natural language collections (empty sets, for example, can produce arbitrary nestings), especially when it comes to infinite sets.

If you want finitism, you need a principled way to select a largest finite number.

Answer by TAG20

However, I find myself appealing to basic logical principles like the law of non-contradiction.

The law of non contradiction isn't true in all "universes" , either. It's not true in paraconsistent logic, specifically.

TAG-30

Yes, and Logan is claiming that arguments which cannot be communicated to him in no more than two sentences suffer from a conjunctive complexity burden that renders them “weak”.

@Logan Zoellner being wrong doesn't make anyone else right. If the actual argument is conjunctive and complex, then all the component claims need to be high probability. That is not the case. So Logan is right for not quite the right reasons -- it's not length alone.

That’s not trivial. There’s no proof that there is such a coherent entity as “human values”, there is no proof that AIs will be value-driven agents, etc, etc. You skipped over 99% of the Platonic argument there.

Many possible objections here, but of course spelling everything out would violate Logan’s request for a short argument.

And it wouldn't help anyway. I have read the Sequences , and there is nothing resembling a proof , or even strong argument, for the claim about coherent human values. Ditto the standard claims about utility functions, agency , etc. Reading the sequence would allow him to understand the LessWrong collective, but should not persuade him.

Whereas the same amount of time could, more reasonably, be spent learning how AI actually works.

Needless to say, that request does not have anything to do with effectively tracking reality,

Tracking reality is a thing you have to put effort into, not something you get for free, by labelling yourself a rationalist.

The original Sequences have did not track reality , because they are not evidence based -- they are not derived from academic study or industry experience. Yudkowsky is proud that they are "derived from the empty string" -- his way of saying that they are armchair guesswork.

His armchair guesses are based on Bayes,von Neumann rationality, utility maximisation, brute force search etc, which isnt the only way to think about AI, or particularly relevant to real world AI. But it does explain many doom arguments, since they are based on the same model -- the kinds of argument that immediately start talking about values and agency. But of course that's a problem in itself. The short doomer arguments use concepts from the Bayes/VonNeumann era in a "sleepwalking" way, out of sheer habit, given that the basis is doubtful. Current examples of AIs aren't agents, and it's doubtful whether they have values. It's not irrational to base your thinking on real world examples, rather than speculation.

In addition , they haven't been updated in the light of new developments , something else you have to do to track reality. People other than Yudkowsky have written about AI safety from the perspective of how real world AIs work, but that just makes the overall mass of information larger and more confusing.

where there is no “platonic” argument for any non-trivial claim describable in only two sentence, and yet things continue to be true

You are confusing truth and justification.

@Tarnish

You need to say something about motivation.

@avturchin

There are dozens of independent ways in which AI can cause a mass extinction event at different stages of its existence.

While each may have around a 10 percent chance a priori, cumulatively there is more than a 99 percent chance that at least one bad thing will happen.

Same problem. Yes, there's lots of means. That's not the weak spot. The weak spot is motivation.

@Odd anon

Same problem. You've done nothing to fill the gap between "ASI will happen" and "ASI will kill us all".

TAG10

As other people have said, this is a known argument; specifically, it’s in The Generalized Anti-Zombie Principle in the Physicalism 201 series. From the very early days of LessWrong

Albert: “Suppose I replaced all the neurons in your head with tiny robotic artificial neurons that had the same connections, the same local input-output behavior, and analogous internal state and learning rules.”

I think this proof relies on three assumptions. The first (which you address in the post) is that consciousness must happen within physics. (The opposing view would be substance dualism where consciousness causally acts on physics from the outside.) The second (which you also address in the post) is that consciousness and reports about consciousness aren’t aligned by chance. (The opposing view would be epiphenomenalism, which is also what Eliezer trashes extensively in this sequence.) physical duplicate might do the same, although. that would imply the original's consciousness is epiphenomenal. Which is itself a reason to disbelieve in p-zombies , although not an impossibility proof.

This of course contradicts the Generalised Anti Zombie Principle announced by Eliezer Yudowsky. The original idea was that in a zombie world, it would be incredibly unlikely for an entity's claims of consciousness to be caused by something other than consciousness. "

Excluding coincidence doesn't proved that an entity's reports of consciousness are directly caused by its own consciousness. Robo-Chalmers will claim to be conscious because Chalmers does. It might actually be conscious, as an additional reason, or it might not. The fact that the claim is made does not distinguish the two cases. Yudkowsky makes much of the fact that Robo-Chalmers claim.would be caused indirectly by consciousness -- Chalmers has to be conscious in order to make a computational duplicate of his consciousness -- but at best that refutes the possibility of a zombie world, where entities claim to be conscious, although consciousness has never existed. Robo-Chalmers would still be possible in this world for reasons Yudkowsky accepts. So there is one possible kind of zombie, even given physicalism so the Generalised Anti Zombie Principle is false

(Note that I am talking about computational zombies, or c-zombies, not p-zombies

Computationalism isn't a direct consequence of physicalism. Physicalism has it that an exact atom-by-atom duplicate of a person will be a person and not a zombie, because there is no nonphysical element to go missing. That's the argument against p-zombies. But if actually takes an atom-by-atom duplication to achieve human functioning, then the computational theory of mind will be false, because there CTM implies that the same algorithm running on different hardware will be sufficient. Physicalism doesn't imply computationalism, and arguments against p-zombies don't imply the non existence of c-zombies-duplicates that are identical computationally, but not physically).

@Richard_Kennaway

That sounds like a Chalmers paper. https://consc.net/papers/qualia.html

Load More