Of course, you're still taking sensory inputs as primitives. How do you then evaluate changes to your sensory apparatus?

I agree with the "concept as regularity" concept. You can see that in how computers use network packets to communicate with each other. They don't define a packet as a discrete message from another computer, they just chop it up and process it according to its regularities

This leads to problems trying to point at humans in an AI motivational system though. Which you have to build yourself.... The problem is this. Starting at the level of visual and audio input signals, build a regularity parser that returns a 1 when it apprehends a human, and 0 when it sees apprehends something else. You have to do the following, future proof it so it recognises post/trans humans as humans (else if might get confused when we seem to want to wipe ourselves out). Make sure it is not fooled by pictures, mannequins, answer phones, chat bots.

Basically you have to build a system that can abstract out the computational underpinning of what it means to be human, and recognise it from physical interaction. And not just any computational underpinning, as physics is computational there is tons of physics of our brains we don't care about, such as exactly how we get different types of brain damage from different types of blunt trauma. So you have to build a regularity processor that abstracts what humans think are important about the computational part of humans.

If you understand how it does this, you should be able to make uploads.

We develop an understanding of what it means to be human, through interactions with humans. With a motivational system that can be somewhat gamed by static images and simulations, but one we don't trust fully. This however leads to conflicting notions about humanity. Whether uploads are humans or not, for example. So this type of process should probably not be used for something that might go foom.

Yup. And for preference, it's the same situation, except that there is only one preference (expressed in terms of I/O) and it doesn't depend on observations (but it determines what should be done for each possible observation sequence). As concepts adapt to actual observations, so could representations of preference, constructed specifically for efficient access in this particular world (but they don't take over the general preference definition).

I've kind of wanted to write about the concept-as-regularity thing for a while, but it seems akrasia is getting the best of me. Here's a compressed block of my thoughts on the issue.

Concept-as-regularity ought to be formalized. It is possible to conclude that a concept makes sense under certain circumstances involving other existing concepts that are correlated with no apparent determining factor. Since a Y-delta transformation on a Bayesian network looks like CAR, I'm guessing that the required number of mutually correlated concepts is three. Formalizing CAR would allow us to "formally" define lots of concepts, hopefully all of them. Bleggs and rubes are a perfect example of what CAR is useful for.