Life, sin, disease, redness, maleness and indeed dogness "may" also be like electromagnetism. The English language may also be a fundamental part of the universe and maybe you could tell if "irregardless" or "wanna" are real English words by looking into a microscope or turning your telescope to certain parts of the sky, or maybe by looking at chicken intestines, who knows. I know some people think like this. Stuart Hameroff says that morality may be encoded into the universe at the Planck scale. So maybe that's where you should look for "good", maybe "pleasure" is there as well.

But anyway, research into electromagnetism was done using the scientific method, which means that the hypothesis had to produce predictions that were tested and replicated numerous times. What sort of experiment would you envision for testing something about "inherently pleasurable" arrangements of atoms? Would the atoms make you feel warm and fuzzy inside when you look at them? Or would you try to put that pattern into different living creatures and see if they react with their normal joyful reactions?

I don't like the expression "carve reality at the joints", I think it's very vague and hard to verify if a concept carves it there or not. The best way I can imagine this is that you have lots of events or 'things' in some description space and you can notice some clusterings, and you pick those clusters as concepts. But a lot depends on which subspace you choose and on what scale you're working... 'Good' may form a cluster or may not, I just don't even know how you could give evidence either way. It's unclear how you could formalize this in practice.

My thoughts on pleasure and the concept of good is that your problem is that you're trying to discover the sharp edges of these categories, whereas concepts don't work like that. Take a look at this LW post and this one from Slatestarcodex. From the second one, the concept of a behemah/dag exists because fishing and hunting exist.

Try to make it clearer what you're trying to ask. "What is pleasure really?" is a useless question. You may ask "what is going on in my body when I feel pleasure?" or "how could I induce that state again?"

You seem to be looking for some mathematical description of the pattern of pleasure that would unify pleasure in humans and aliens with totally unknown properties (that may be based on fundamentally different chemistry or maybe instead of electomagnetism-based chemistry their processes work over the strong nuclear force or whatever). What do you really have in mind here? A formula, like a part of space giving off pulses at the rate of X and another part of space at 1 cm distance pulsating with rate Y?

You may just as well ask how we would detect alien life at all. And then I'd say "life" is a human concept, not a divine platonic object out there that you can go to and see what it really is. We even have edge cases here on Earth, like viruses or prions. But the importance of these sorts of questions disappears if you think about what you'd do with the answer. If it's "I just want to know how it really is, I can't imagine doing anything practical with the answer" then it's too vague to be answered.

This is part of the Hard Problem of Consciousness: why is there any such thing and how does it work? It is Hard because we cannot even see what a solution would be. Even if we discovered patterns of neural activity or anything else that reliably and in great detail matched up with the experience, it seems that that still wouldn't tell us why there is such a thing as that experience, and would not suggest any test we could apply to a synthetic imitation of the patterns.

(7) If we met an alien life-form, how could we tell if it was suffering?

The world is already full of alien life-forms -- that is, life-forms radically different from yourself. How do you decide, and how should you decide, which of the following suffers? A human being with toothache; a dog that has been hit by a car; a mouse bred to grow cancers; a wasp infected by a fungus that is eating up its whole body and sprouting from its surface; a caterpillar paralysed and being eaten alive by the larvae of that wasp; a jellyfish stranded on the beach that a playful child has thrust its spade into; a fish dying from the sting of a jellyfish; a tree with the sort of burr that wood carvers prize for its ornamental patterns; parched grass in a drought. And, for that matter, a cliff face that has collapsed in a great storm; tectonic plates grinding together; a meteor burning up in the atmosphere.

Surely neurological processes are "arrangements of particles" too, though.

Processes are not "arrangements", it's a dynamic vs static difference.

Good is a complex concept, not an irreducible basic constituent of the universe. It's deeply rooted in our human stuff like metabolism (food is good), reproduction (sex is good), social environment (having allies is good) etc. We can generalize from this and say that the general pattern of "good" things is that they tend to reinforce themselves. If you feel good, you'll strive to achive the same later. If you feel bad, you'll strive to avoid feeling that in the future. So if an experience makes more of it then it's good, otherwise it's bad.

Note that we could also ask: "Is there a general principle to be found with regard to which patterns within conscious systems innately feel like smelling a rose, or isn't there?" We could build rose smell detecting machines in various ways. How can you say that one is really having the experience of smelling it while another isn't?

This all seems to be about the "qualia" problem. Take another example. How would you know if an alien was having the experience of seeing the color red? Well, you could show it red and see what changes. You could infer it from its behavior (for example if you trained it that red means food - if indeed the alien eats food).

Similarly you could tell that it's suffering when it does something to avoid an ongoing situation, and if later on it would very much prefer not to go under the same conditions ever again.

I don't think there is anything special about the actual mechanism and neural pattern that expresses pain or suffering in our brains. It's that pattern's relation to memories, sensory inputs and motor outputs that's important.

Probably you could even retrain the brain to consider a certain fixed brain stimulus to be pleasure even though it was previously associated with pain. It's like putting on those corrective glasses that turn the visual input by 180° and the brain can adapt to that situation and the person is feeling normal after some time.

Pleasure is not a static "arrangement of particles". Pleasure is a neurological process.

You can't find a "pleasure pattern" that's fully generalized. The information is always contextual.

This isn't a perfect articulation of my objections, but this is a difficult subject.

Off-topic, but I notice that this post, according to the time-stamp, was apparently posted on March 1, 2015. There are comments attached to it, however, dating from 2013. Does anyone know why this is?

These are great questions. I'm not sure they have answers. But they seem extremely pertinent to making a good AGI.

Tegmark's paper here: http://arxiv.org/pdf/1409.0813.pdf seems to be poking in the same direction.

Neglecting these questions is, IMO, tantamount to moral relativism or nihilism.