"Every set of numbers has a least element" clearly does NOT define the natural numbers. Consider N U {-1}.

I'm afraid I won't have time to give you more help. There's a short summary of each sequence under the link at the top of the page, so it won't take you forever to see the relevance.

EDIT: you're wondering elsewhere in the thread why you're not being well received. It's because your post doesn't make contact with what other people have thought on the topic.

While I don't have too much experience to back this up, I think it is probably a lot of things I'm familiar with, elaborated at length, with perhaps a few insights sprinkled in.

Yes, I don't particularly like the way the sequences are written either :/ But I think the kind of thing you're talking about in this post is the sort of topic they address. LW Wiki pages are often better, e.g. see this one:

if a p-zombie is atom-by-atom identical to a human being in our universe, then our speech can be explained by the same mechanisms as the zombie's, and yet it would seem awfully peculiar that our words and actions would have one entirely materialistic explanation, but also, furthermore, our universe happens to contain exactly the right bridging law such that our experiences are meaningful and our consciousness syncs up with what our merely physical bodies do. It's too much of a stretch: Occam's razor dictates that we favor a monistic universe with one uniform set of laws.

I see this as compatible with my reply to skeptical_lurker above.

My point is: how do you evaluate if something has preferences? How do you disambiguate preferences from statements like "I prefer __"? Clearly we DO distinguish between these.

Agreed. I don't have any easy answer to this question. It's kind of like asking the question "if someone is ill or injured, how do you fix them?" It's an important question worthy of extensive study (at least insofar as it's relevant to whatever ethical question you're currently being presented with).

And it's possible that you and I would disagree on how to carve reality in to that which has preferences we consider meaningful vs that which doesn't. Occam's Razor only applies to the territory, not the map, so there's no penalty for us drawing our boundaries in as complicated & intricate a way as we like (kind of like the human-drawn country boundaries on real maps).

Suppose all information processing is inextricably linked to qualia. Now I suppose there is information processing in rocks of a form, in the equations of themodynamics, motion etc that govern the rocks behaviour. But qualia does not imply self-awareness (1), and there's no way you can communicate with the rock. Qualia also doesn't imply emotions (2), and if there is neither self-awareness nor emotions then I don't see why there need be any moral considerations.

As to determining the truth of Panpsychism and categorising which things have emotions, self awareness etc, I shall defer this problem to future superintelligences. Additionally, a CEV AI should devote a lot of resources to humans regardless of whether panpsychism is true, because most people don't believe in Peter Singer style altruism.

1 because (a) people who meditate for many years or take a large does of dissociative drugs can experience ego-death, where they stop conceptualising a self, but they still experience qualia. (b) most animals are not self-aware, yet intuitions and occam's razor tell me that they still experience qualia

2 some people experience emotional blunting, but while the world may seem grey emotionally, yet they still experience qualia. Additionally, squid do not have emotions, and again I believe they still have qualia.

EDIT: as well as lacking self-awareness and emotions, rocks also lack agency. The question of what to do with a human, who due to various incurable diseases, lacks self-awareness, emotions and agency is left as an excersize for the reader.

Have you read A Human's Guide to Words?

If you have: taboo "consciousness," because the definition itself is what you are uncertain about. What is it that you care if atoms have?

In my opinion, it looks like there is no physical property at stake here - you are instead worried about if atoms have you-care-about-them substance. Which is of course not an actual substance in the atoms - your question ultimately points back to your own ideas and preferences.

This is your own prerogative, but I think if you try to let go of the question of "consciousness" for now until you have a better idea of what physical properties correspond to it, and just ask yourself if you care about atoms for their own sake, you probably do not, and thus can stop worrying about it.

Panpsychism seems like a plausible theory of consciousness.

What's the best argument you've seen for it? I don't find it plausible myself.

What do you think of the reductionism sequence? E.g. this post might be relevant.

My interpretation of the "expanding circle" might be something like: it'd be a good thing if things with preferences increasingly found themselves preferring that the preferences of other things with preferences were also achieved. If something doesn't have preferences, I'm not that concerned about it.

So how can the universe "enjoy itself" as much as possible before the big crunch (or before and during the heat death)*.

Maybe read the Fun Theory sequence?

It's an interesting idea, but it's not at all new. Most moral philosophers would agree that certain experiences are part (or all) of what has value, and that the precise physical instantiation of these experiences does not necessarily matters (in the same way many would agree on this same point in philosophy of consciousness).

There's a further meta-issue which is why the post is being downvoted. Surely is vague and maybe too short, but it seems to have the goal of initiating discussion and refining the view being presented rather than adequately defending or specifying it. I have posted tentative discussions - much more developed than this one - in meta-ethics or other abstract issues in ethics directly related to rationality and AI-safety, and I wasn't exactly warmly met. Given that much of the central problems being discussed here are within ethics, why the disdain for meta-ethics? Of course, it might as well just be a coincidence or that all those posts were fundementaly flawed in a obvious way.

During the end of a drive there would either be/not be a configuration of particles in the shape of a paper ticket memorializing your transgression of the law. And if not that, there is a configuration of particles in the heads of the law enforcement officials recalling your transgression and planning on writing you a ticket or whatever. Any universe-wide configuration of particles contains the history of all of the events preceding it, even if they are opaque to us, because the possibility space of particle configurations is (probably?) larger than utility relevant histories.

In AI research, intelligent agents typically have a clear-cut and well-defined final goal, e.g., win the chess game or drive the car to the destination legally. The same holds for most tasks that we assign to humans, because the time horizon and context is known and limited. (...) a truly well-defined goal would specify how all particles in our Universe should be arranged at the end of time.

We typically only care about the arrangement of particles at the end of the task, because that is the nature of the simple tasks we usually use machines for today. Actually, even that is not true: when "driving the car to destination legally" we care not only about the arrangement of the particles of the car at the end of the trip, but also about what happened on the way -- that's what "legally" means here. (Unless we also count "police sending us tickets" as particles. But I guess the car is supposed to follow the laws even when the police does not look.)

We can define "journey" goals e.g. by calculating score at each time interval, and trying to maximize the sum or the average (or some other function) of all the intervals. This can make sense even if we don't know how long the task will last.

treat experience as inherently positive and not try to distinguish between positive and negative experiences.

This sounds wrong. But I am not even sure what exactly would we measure here, if both positive and negative experience count the same. Is it the intensity of the experience (in either direction) which counts? (That is, would you rather be tortured than bored? Would you rather be tortured really painfully than enjoying a mild pleasure?) Or is it duration of the experience? (That is, we want to maximize the subjective time of sentient beings, regardless of what happens during the time? Would you rather live 1001 years in hell than 1000 years in heaven?)