What I mean by preference is a valuation of how I want the world to be. It's not about cognitive ritual, although cognitive ritual, as a part of the world, may also be mentioned there. Preference is not the sort of thing that does anything, it is a statement of what I think should be done. Through the activity of the mind, the way preference is may influence other things, and conversely, other things may influence preference, as in the case of wireheading, for example.
I don't understand what you mean by "cognitive ritual".
This is a statement in connotation opposite to one that triggered my comment in the first place, see here. How do you recognize which motivations you choose to identify with, and which you don't? I guess in this model, the criterion may be said to derive from that very preference stuff.
I couldn't make heads or tails of that comment, sorry. I'm not entirely sure I understand what you wrote here, either, except that it sounds like you think we "choose" to identify with things. My observation is that choice is not the default -- we have the ability to choose, but mostly, we don't use it, and when we think we are, we are mostly lying to ourselves.
This doesn't much connect to standard theories or intuition, for the same reason that relativity doesn't: it's correct over a wider range of conditions than our default intuitions. If you view minds through a mechanical lens, their behaviors don't require such complex explanations.
How is this a fact? From my perspective, we are groping in the dark at this point, so any statement should either be intuitive, as raw material to build upon, generated from being primed by a representative sample of data, to give any chance of showing true regularities, or study the few true regularities that can be supported.
I say that it's a fact our preferences are barely related to our motivations because it's trivial to show that they function independently -- you've pointed this out yourself. That most people fail to change their motivation by modifying their preferences is more than sufficient to demonstrate the lack of connection in practice between these two brain functions. (See also the near/far distinction.)
I don't understand the relation between preference, motivation, shouldness, influence, and facts that you are making in the above quoted sentence.
By "should" I mean, expecting that merely having a preference will automatically mean you have corresponding motivation, or that the lack of ability to enforce your preference over your motivation equals a personal failure -- it merely reflects the "design" parameters of the systems involved. There is no evolutionary reason for us to control over our motivations, since they exist to control us -- to shape us to the world we find ourselves in.
What's a 'schema', what kind of object is this motivation thing that can be created or deleted?
By schema here, I'm referring to "near" vs. "far" thinking. Action versus abstraction.
A motivation is simply an emotional response attached to an outcome or behavior, through conditioning or simple association.
Are there many of them if one can be created and deleted?
Yep.
What role do they play in the cognitive algorithm?
They drive the planning process, which we experience as "motivation". See, for example, the video I and others have linked here before, which demonstrates how to induce a (temporary) motivation state to clean your desk. There is a lot of deep theory behind that video, virtually none of which is present in the video.
If there is no relation between preferences and these motivation instances, what is the role played respectively by preference and emotion in the overall algorithm?
If you really care about deep understanding of the "cognitive algorithm", you would be well advised to read "NLP Volume I", which explains the model I use quite well. As its subtitle calls it, "the study of the structure of subjective experience" -- i.e., what algorithms feel like from the inside.
The motivation video I made demonstrates one simple algorithm ("strategy" in NLP lingo) that is conveyed in terms of sensory representation ("near" thinking) steps. This is because most of our actual cognitive processing consists of manipulating sensory data, both in and out of consciousness. Verbal processing gives us flexibility and suggestibility, but a huge part of our outward verbalization is devoted to making up plausible explanations and things that make us sound good. And it is driven by our motivations (including hard-wired and trained status motivations), rather than being a source of motivations.
The distinction can be easily seen in my video, as I demonstrate using verbal thinking merely to suggest and "lead" the near system to evoke certain sensory representations in visual and kinesthetic form, rather than by trying to "talk" one's self into doing something through logic or slogan.
Btw, a lot of the disconnect that you're experiencing from my writing is simply that if you care more about theory than practice, you need to read a hell of a lot more than what I write, to understand what I'm writing about.
I've been studying NLP in my spare time for around 20 years now, and there is absolutely no way I can teach that entire field of study in off-hand comments. Since most people are more interested in practice than theory, I focus my writing to have the least amount of theory that's needed to DO something, or at least come to an understanding of why what you're already doing doesn't work.
If you insist on implementation-quality theory, and you don't "get" representational systems and strategies as the primary model of all behavior (internal as well as external), you're not going to "get" what I write about, because I presuppose that that model is the closest thing we have to a functional theory of mind, from a practical-results perspective. There is nothing in mainstream cognitive psychology that remotely approaches the usefulness of NLP as a model of subjective experience and behavior, which likely means there's nothing approaching its accuracy as an operational model.
(Disclaimer: Popular depictions of NLP are ridiculously shallow, so anyone who hasn't read "NLP, Volume I" or "The Structure Of Magic I", stands a very strong chance of not even remotely knowing what NLP actually is. Even some supposedly-certified "practitioners" have no clue, treating the theory as something they just had to learn to get their certificate, alas. Having a bit more epistemic hygiene probably would be helpful to the discipline as a whole... but then, you can say that about most fields.)
I say that it's a fact our preferences are barely related to our motivations because it's trivial to show that they function independently -- you've pointed this out yourself. That most people fail to change their motivation by modifying their preferences is more than sufficient to demonstrate the lack of connection in practice between these two brain functions.
Being separate is far from the same thing as being independent, or having no connection with each other. It is only grounds for introducing a concept, for making a distinction.
Also, at this poin...
Related to: Joy in the Merely Real, How An Algorithm Feels From Inside, "Science" As Curiosity-Stopper
Your friend tells you that a certain rock formation on Mars looks a lot like a pyramid, and that maybe it was built by aliens in the distant past. You scoff, and respond that a lot of geological processes can produce regular-looking rocks, and in all the other cases like this closer investigation has revealed the rocks to be completely natural. You think this whole conversation is silly and don't want to waste your time on such nonsense. Your friend scoffs and asks:
"Where's your sense of mystery?"
You respond, as you have been taught to do, that your sense of mystery is exactly where it should be, among all of the real non-flimflam mysteries of science. How exactly does photosynthesis happen, what is the relationship between gravity and quantum theory, what is the source of the perturbations in Neptune's orbit? These are the real mysteries, not some bunkum about aliens. And if we cannot learn to take joy in the merely real, our life will be empty indeed.
But do you really believe it?
I loved the Joy in the Merely Real sequence. But it spoke to me because it's one of the things I have the most trouble with. I am the kind of person who would have much more fun reading about the Martian pyramid than about photosynthesis.
And the one shortcoming of Joy in the Merely Real was that it was entirely normative, and not descriptive. It tells me I should reserve my sense of mystery for real science, but doesn't explain why it's so hard to do so, or why most people never even try.
So what is this sense of mystery thing anyway?
I think the sense of mystery (sense of wonder, curiosity, call it what you want) is how the mind's algorithm for determining what problems to work on feels from the inside. Compare this to lust, how the mind's algorithm for determining what potential mates to pursue feels from the inside. In both cases, the mind makes a decision based on criteria of its own, which is then presented to the consciousness in the form of an emotion. And in both cases, the mind's decision is very often contrary to our best interest - as anyone who's ever fallen for a woman based entirely on her looks can tell you.
What sort of stuff makes us curious? I don't have anything better than introspection to go on, but here are some thoughts:
1. We feel more curious about things that could potentially alter many different beliefs.
2. We feel more curious about things that we feel like we can solve.
3. We feel more curious about things that might give us knowledge other people want but don't have.
4. We feel more curious about things that use the native architecture; that is, the sorts of human-level events and personal interactions our minds evolved to deal with.
So let's go back and consider how the original example - a pyramid on Mars versus photosynthesis - fits each of these criteria:
The pyramid on Mars could alter our worldview completely1. We'd have to rework all of our theories about ancient history, astronomy, the origin of civilization, maybe even religion. Learning exactly how photosynthesis works, on the other hand, probably won't make too big a difference. I assume it probably involves some sort of chemistry that sounds a lot like the other chemistry I know. I anticipate that learning more about photosynthesis wouldn't alter any of my beliefs except those directly involving photosynthesis and maybe some obscure biochemical reactions.
Pseudoscience and pseudohistory feel solveable. When you're reading a good pseudoscience book, it feels like you have all the clues and you just have to put them together. If you don't believe me, Google some pseudoscience. You'll find hundreds of webpages by people who think they've discovered the 'secret'. One person who says the pyramid on Mars was made by Atlanteans, another who says it was made by the Babylonian gods, another who says it was made by God to test our faith. On the other hand, I know I can't figure out photosynthesis without already being an expert in chemistry and biology. There's not that tantalizing sense of "I could be the one to figure this out!"
Knowing about a pyramid on Mars means you know more than other people. Most of humankind doesn't think there are any structures on Mars - the fools! And if you were to figure it out, you'd be...one of the greatest scientists ever. The one who proved the existence of intelligent life on other planets. It'd be great! In comparison, knowing about photosynthesis makes you one of a few thousand boring chemist types who also know about photosynthesis. Even if you're the first person to discover something new about it, the only people likely to care are...a few thousand boring chemist types.
And the pyramid deals in human-level problems: civilizations, monuments, collapse. Photosynthesis is a matter of equations and chemical reactions; much harder for most people.
Evolutionarily, all these criteria make sense. Of course you should spend more time on a problem if you're likely to solve it and the solution will be very important. And when you're a hunter-gatherer, all your problems are going to be on the human level, so you might as well direct your sense of mystery there. But the algorithm is unsuited to modern day science, when interesting discoveries are usually several inferential distances away in highly specialized domains and don't directly relate to the human level at all.
Again, compare this to lust. In the evolutionary era, mating with a woman with wide hips was quite adaptive for a male. Nowadays, with the advent of the Caesarian section, not so much. Nowadays it's probably most important for him to choose a mate whom he can tolerate for more than a few years so he doesn't end up divorced. But the mental algorithms whose result outputs as lust don't know that, so they end up making him weak-kneed for some wide-hipped woman with a terrible personality. This isn't something to feel guilty about. It's just something he needs to be wary of and devote some of his willpower resources toward fighting.
The practical take home advice, for me at least, is to treat curiosity in the same way. For a while, I felt genuinely guilty about my attraction to pseudohistory, as if it was some kind of moral flaw. It's not, no more than feeling lust towards someone you don't like is a moral flaw. They're both just misplaced drives, and all you can do is ignore, sublimate, or redirect them2.
The great thing about lust is that satisfying your unconscious and conscious feelings don't have to be mutually exclusive. Sometimes somebody comes around who's both beautiful and the sort of person you want to spend the rest of your life with. Problem solved. Other times, once your conscious mind commits to someone, your unconscious mind eventually starts coming around. These are the only two solutions I've found for the curiosity problem too.
The other practical take home advice here is for anyone whose job is educating others about science. Their job is going to be a lot easier if they can take advantage of this sense of mystery. The best science teachers I know do this. They emphasize the places where science produces counterintuitive, worldview-changing results. They present their information in the form of puzzles just difficult enough for their students to solve with a bit of effort. They try to pique their students interest with tales of the unusual or impressive. And they try to use metaphors to use the native architecture of human minds: talking about search algorithms in terms of water flowing downhill, for example.
I hope that any work that gets done on Less Wrong involving synchronizing conscious and unconscious feelings and fighting akrasia can be applied to this issue too.
Footnotes:
1: The brain seems generally bad at dealing with tiny probabilities of huge payoffs. It may be that the payoff measured in size of paradigm shift from any paranormal belief being true is just so high that people aren't very good at discounting for the very small percent chance of it being true.
2: One big question I'm still uncertain about: why do some people, despite it all, find science really interesting? How come this is sometimes true of one science and not others? I have a friend who loves physics and desperately wants to solve its open questions, but whose eyes glaze over every time she hears about biology - what's up with that?