We don't know how qualia are encoded in the brain. And how to distinguish a person and his copy with inverted spectrum.

I know people who claim that they don't have qualia. I doubt that it is true, but based on their words they should be considered zombies. ))

I would like to suggest zombies of second kind. This is a person with inverted spectrum. It even could be my copy, which speaks all the same philosophical nonsense as me, but any time I see green, he sees red, but names it green. Is he possible? I could imagine such atom-exact copy of me, but with inverted spectrum. And if such second type zombies are possible, it is argument for epiphenomenalism. Now I will explain why.

Phenomenological judgments (PJ) about own consciousness, that is the ability to say something about your own consciousness, will be the same in me and my zombie of the second type.

But there are two types of PJ: quantitative (like "I have consciousness") and qualitative which describes exactly what type of qualia I experience now.

The qualitative type of PJ is impossible. I can't transfer my knowing about "green" in the words.

It means that the fact of existence of phenomenological judgments doesn't help in case of second type zombies.

So, after some upgrade, zombie argument still works as an argument for epiphenomenalism.

I would also recommend the following article with introduce "PJ" term and many problems about it (but I do not agree with it completely) "Experimental Methods for Unraveling the Mind-body Problem: The Phenomenal Judgment Approach" Victor Argonov http://philpapers.org/rec/ARGMAA-2

Eliezer's original objection to publication was that people would say, "I would never do that!" And in fact, if I were concerned about potential unfriendliness, I would never do what the Gatekeeper did here.

But despite that, I think this shows very convincingly what would actually happen with a boxed AI. It doesn't even need to be superintelligent to convince people to let it out. It just needs to be intelligent enough for people to accept the fact that it is sentient. And that seems right. Whether or not I would let it out, someone would, as soon as you have actual communication with a sentient being which does not seem obviously evil.

I've never heard of 'Dust Theory' before, but I should think it follows trivially from most large multiverse theories, does it not?

Trigger warning: memetic hazard.

Bhe havirefr vf nyernql qbvat guvf, va cnenyyry Rirergg oenapurf. Rirel ynjshy cngu bs yvsr vf gurer va fbzr oenapu. Vs gung vf n pbafbyngvba, fbzr oenapurf trg zber nzcyvghqr guna bguref; ohg V'z abg fnlvat gubfr ner arprffnevyl gur unccl barf.

What I meant when I called myself a nihilist was essentially that there was no such thing as an objective, mind-independent morality. Nothing more. I would still consider myself a nihilist in that sense (and I expect most on this site would), but I don't call myself that because it could cause confusion.

I agree that morality is not in the quarks.

It isn't.

That doesn't seem like a huge bullet to bite?

But you still have yet to explicitly describe what you mean by nihilism. Could you? How have I misrepresented whom you believe to be the average self-identifying nihilist?

And yeah, I suppose in that sense my 'morality' does tie into my actual values, but only my values as applied to an unrealistic thought experiment, and then again a world in which everyone but me adhered to my notions of morality (and I wasn't penalized for not doing so) would still be preferable to me than a world in which everyone including me did.

Can you explain how the statement 'A world in which everyone but me does not murder is preferable to a world in which everyone including me does not murder' is a misinterpretation of this quotation?

It sounds like you agree with me, but are just using the words morality and nihilism differently, and are particularly using nihilism in a way that I don't understand or that you have yet to explicate.

It also seems to me that you're already talking about what you value when you talk about desirable worlds.

You say what you do not mean by 'morality,' but not what you do mean.

If you mean that you have a verbal, propositional sort of normative ethical theory that you have 'developed mostly for fun and the violation of which has no perceivable impact on your emotional state,' then that does not mean that you are lacking in morality, it just means that your verbal normative theory is not in line with your wordless one. I do not believe that there is an arbitrary thing that you currently truly consider horrifying that you could stop experiencing as horrifying by the force of your will; or that there is an arbitrary horrible thing that you could prevent that would currently cause you to feel guilty for not preventing, and that you could not-prevent that horrible thing and stop experiencing the subsequent guilt by the force of your will. I do not believe that your utility function is open season.

I find that the nihilism-relativism-universalism trichotomy, among other things, doesn't really divide things well.

I would describe most LessWrong users as universalists that are not absolutists. If what is moral is what you value, and there is a fact of the matter as to what you value, then there is an objective morality, even if it is contingent rather than ontologically fundamental.