But as you said, we can't actually recompute everything. No time. So the exhortation to "think!" can't possibly be followed in more than a small fraction of the cases.
The best we can do is to occasionally recompute certain items. And, if the re-computation is significantly at odds with the cached result, communicate this to others, who are likely to have the same cached result. We can do this in parallel. You can recompute a few things, I'll recompute a few things, and thousands of others are meanwhile recomputing a few things. Occasionally someone may have a significantly different result, which he'll hopefully communicate to others. The number of significantly different results will hopefully be only a small fraction of the number of recomputed results, which might bring the sharing of different results within the realm of possibility. For example, if there are 100 of us, and each recomputes 10 results, then collectively we recompute 1000 results (assuming no overlap). Only 1 out of every 100 recomputed results might be different from the cached result. So we only need to share among ourselves the 10 significantly different recomputed results. That is pretty easy, and we will in effect have done an overhaul of 1000 cached results, at the price of only 10 recomputations each and 10 received communications each (and one transmitted communication for each person who computed a new result). Seems doable.
What seems to be blatantly forgotten is that people believe themselves to be too busy for "meditation" (as in sitting down and thinking, not necessarily in a religious way) which is coincidentally exactly the process for "clearing the cache". Because we run around all day working and consuming entertainment instead of sitting on a hill watching sheep eat grass meditation has simply lost it's allure. It's a sad statement really because with only 10 minutes of meditation you can free up 1-10 cached thoughts, which when practiced over the course of a year would result in up to 3,650 "cached thoughts" being revisited. Not as optimal as the above solution, but humans wouldn't submit to that sort of computer-like efficiency anyway.
bw: Could you please at least provide a citation or reference for us ignorant fools who don't understand how death gives meaning to life?
I'll have to agree with the diagnosis of Stockholm Syndrome.
Yeah, and if you haven't spent months or, better, years studying astrology you're in no position to discuss that either, especially dismissively. And if you haven't been personally abducted by aliens you have you shouldn't... [etc] /sarcasm
Surely if only the greatest thinkers thought it, everyone else who holds it to be true has it cached?
"Can you imagine having to program using 100Hz CPUs, no matter how many of them you had?"
No, it would be very difficult. But one thing I'm wondering is what's the instruction set of the neuron? I'm probably taking the analogy too far. Is it more advanced then add/sub/mult/div ?
The question is not whether it is a cached tought but whether it is a good thought. And what I claim is that it is both good and extremely difficult to understand precisely because of our natutal bias to avoid death. As for references, I suppose it is a central thought in continental philosophy since Hegel: Heidegger and Jonas but you can find it elsewhere, even as far away from existentialism as Mayr or Maturana.
So how much of the brains advanced nature comes from slower processors with better instruction sets and how much comes from network effects (both spatial and temporal)?
FWIW, As far as caching goes, I've noticed cache failures many, many times in my life. Mostly when I'm doing something that's 99% routine but for some reason I should be changing that last 1% and forget to. For example, if I'm supposed to run an errand on the way home, it's not uncommon for me to forget the errand. I leave work, think the goal is home and pull the set route from my brain....
"Death gives meaning to life" reminds me of this:
http://tvtropes.org/pmwiki/pmwiki.php/Main.WhoWantsToLiveForever
1."It's a good guess that the actual majority of human cognition consists of cache lookups.
This thought does tend to go through my mind at certain times."
A funny joke...as if the idea expressed in the first sentence may itself have been cached.
2."Raised by silent robots" is a catchy phrase. Did you make it up?
I find this phrasing misleading. "False as X" can mean the same thing "as false as X."
"Operations of thought are like cavalry charges in battle – they are strictly limited in number, they require fresh horses, and must only be made at decisive moments." —Whitehead
By the evidence from physics we can state with assuredness that there is no material, mechanical explaination for all phenomena, yet few seem to be able to accept the results of the most proven scientific theory in the history of mankind.
Sorry, I don't know what you're talking about.
Is caching the best mental model of how these jillions of "100hz processors" operate?
An alternate: lossy decompression. Rather like, for instance, how dna information is expressed during an individual's life. (And, one cannot help but suspect, at a much larger scale than that of the lives of individuals.)
A reason to prefer "lossy compression" over "caching": "Caching" leads one to believe that the information is cached without loss. And, one tends to look around to find where the uncompressed bits can be stored.
But, I'll admit I've failed to put together the pieces of a general intelligence machine using a lossy compression model. So maybe it's a bogus model, too.
Has anyone built the equivalent of a Turing machine using processor count and/or replicated input data as the cheap resource rather than time?
That is, what could a machine that does everything in one step do in the way of useful work? With or without restrictions on how many replications of the input data there are going in and where the output might come out?
OK, OK. "Dude, what are you smoking?", right? :)
Felix: Yes, for example see http://en.wikipedia.org/wiki/NC_%28complexity%29
By the evidence from physics we can state with assuredness that there is no material, mechanical explaination for all phenomena, yet few seem to be able to accept the results of the most proven scientific theory in the history of mankind. When did this physics breakthrough happen? Or are you referring to this?
tggp The breakthrough I'm refering to started in 1900 with Max Planck. This ended in around 1930 with what is now called quantum physics. If you go to the Nobel prize web site and read Max Born's acceptance sppech you'll get a good flavor of this. Also Henry Stapp has written numerous papers along these lines.
You seem to shy away from the obvious conclusion of your otherwise excellent post. Our slow brains are entirely unable to do much reasoning from first principles, therefore we ought to pay strict attention to such received ideas that have stood the test of time, and have been culturally cached. "Love isn't rational" strikes me as an excellent example. If our rationality is bounded, as it certainly is, then it is often rational to not try to think things out from first principles, but accept the evolved memes of the surrounding culture. You may...
Where do you think ancient wisdom comes from, mtraven? From still more ancient wisdom? I've tried to rethink a few things myself, and though I've gone astray from time to time, I wouldn't have it any other way. Not for anything in the world. Sometimes you need a stronger weapon than your ancestors have forged, you see. Tsuyoku naritai!
Eliezer - there is one additional input to surviving ancient wisdom that goes beyond the thought that the ancients put into it, and that is the simple fact of its survival. Even if people came up with an idea for bad reasons, that idea may nevertheless be a good one and may survive on that account. If it survives, then it may be a good idea even though nobody knows why, and even though nobody ever knew why.
I make no recommendation on this basis, I simply point out that there can be more to ancient wisdom than what ancient minds put into it, and an attempt ...
Douglas, this is difficult because you appear to prefer to allude to your position rather than state it.
Quantum mechanics, at least according to some ways of interpreting it, does indeed say that some events don't have any explanation beyond "that's the way it happened to go". So far, so good; but what does that have to do with whether the mind and the brain are the same thing? (Actually, I think physicalists would generally say not "the mind and the brain are the same thing" but something more like "the mind is something the brain...
g- I'm saying that the need to explain your thinking by means of brain processes assumes something about the situation that may not be true. I'm not saying that such a research project is doomed to failure, or violates the laws of physics, just that it is not the only explaination that would agree with what has been discovered in physics. I would further say that when the physicists overcame the idea that there must be a material,mechanical explaination for all the phenomena they were studying we got the most validated scientific theory in history. Som...
Quantum mechanics did not result from overcoming the idea that there must be a material, mechanical explanation for all the phenomena physicists study.
What about quantum mechanics gives us any reason to think that there's anything wrong with Eliezer's commitment to understanding minds in terms of brains?
And could you give a specific example of a difficulty in neuroscience or philosophy that results from a commitment to understanding minds in terms of brains? (I find it easier to think of ones that come from a commitment to not understanding minds in terms of brains.)
Yes, QM came about because of the recognition that classical physics is wrong. (I would take issue with some details of your one-sentence summary, but it doesn't matter.) But then you leap from there to "the recognition that there is no material, mechanical explanation of all phenomena", which is something entirely different.
Bell's inequality and Aspect's experiments demonstrating its violation don't say that there is no material, mechanical explanation of all phenomena. They place limits on what sorts of material, mechanical explanation there mi...
There was a wonderfully illustrative story which I thought I had bookmarked, but couldn't re-find: it was the story of a man whose know-it-all neighbor had once claimed in passing that the best way to remove a chimney from your house was to knock out the fireplace, wait for the bricks to drop down one level, knock out those bricks, and repeat until the chimney was gone. Years later, when the man wanted to remove his own chimney, this cached thought was lurking, waiting to pounce... As the man noted afterward - you can guess it didn't go well - his neighbor was not particularly knowledgeable in these matters, not a trusted source. If he'd questioned the idea, he probably would have realized it was a poor one.
That story was published in Fine Homebuilding a few years ago.
I happen to know because...I was the idiot who tore down a chimney from a bottom on the advice of my neighbor, and after telling the story to some acquaintances at a timber framing class up in Vermont and being told "you should write that up and send it to Fine Homebuilding", I did.
I've got a copy of the story hanging off my blog.
Anyway, love the blog. Keep up the good work.
I really enjoyed your post! I would say we cache things we've reasoned out ourselves as well. Say you do a mathematical proof for the pythagorean theorum. At the end of the proof, you might feel you really understand the theory, but the next year, or next day even, you have completely forgotten the steps you used to do the proof. You might be able with great concentration extrapolate them again, but you still believe the theory without recalculating it from scratch. You remember being convinced in the past, and you trust your past self's judgment. I ...
I don't think I know of anyone who believes that everything is explicable in terms of causally things that have mass and exist as solid, liquid or gas, still less that everything must be. And I can't imagine how anything in Eliezer's original post suggests that he's insisting on any such limitation.
Neither can I see how this has anything to do with QM (except, I guess, that some versions of QM give us a universe with randomness in it as well as determinism), or with Feynman's comment about machinery. (The fundamental laws known at any time are by definition laws that no one has found any machinery behind. This was just as true of Newton's laws in 1700 as of QM in 2000.)
Thinking in text...
Change your mode of cache usage. The brain has two conflicting tendencies here, which I'll name "contagion" and "cull". The contagion tendency is the way that related mental objects prime each other. The cull tendency is the way that a firm decision suppresses valid alternates. Your motto should be "first contagion, then never quite cull". If you cull first, that's "jumping to conclusions". If you contagion but don't cull, that's called "woolgathering" and "being a ditherer". Bu...
g- The cache thought I'm recognzing as false is that science demands material explainations. When I hear the mind described as the brain, that thought is activated in my thinking. Material, mechanistic = scientific. I don't know what is in your mind or Elizer's. I'm trying to deactivate the thought in my mind. Isn't that the point of the post?
Douglas, you appear to have shifted your ground: originally you said "the explanation of cached thoughts assumes the mind and the brain are the same thing" and "I read someone unhesitatingly repeating a meme and thought", but now you say it's only your own cached thoughts that you're concerned about.
I still have no idea why you think that QM makes any difference to how much science "demands material explanations"; with the definition of "material" that you gave it never did, and with any definition of "material&...
In 1998, I wrote a rec.arts.int-fiction post called "Believable stupidity" (http://groups.google.com/group/rec.arts.int-fiction/ browse_thread/thread/60a077934f89a291/ 3fffb9048965857d?lnk=gst&q=believable+stupidity#3fffb9048965857d) split across 3 lines; rejoin for link)
saying that Eliza, a computer program that matches patterns, and fills in a template to produce a response, always wins the Loebner competition because template matching is more like what people do than reasoning is.
Herb Simon's cognitive psych lectures at Carnegie Mellon always started with this same observation of how slow neurons are. He emphasized how bad we are at reasoning logically and how good we are at associative tasks. His and Allen Newell's work on AI in the early 1960s led to the SOAR project, which models thinking as a big production system that caches effective sequences of inferential steps for later re-use. Simon also used to say that it took about 10 years to accumulate a large enough cache to be considered an expert in something.
It's a good guess that the actual majority of human cognition consists of cache lookups.
Sounds consistent with Jeff Hawkins's memory prediction framework
This isn't some theoretical limit of the human brain; it's just what they've found from testing (or something they just made up, now that I think about it). Whoever they were testing was alive, and was taking full advantage of their soul.
Strangely, I have a cached thought of, "That's bullshit." This pings almost everything I hear said by people in a particular verbal/non-verbal pattern. For some reason, when someone says something in a manner that matches this verbal/non-verbal pattern I think, "That's bullshit." It doesn't even matter what they are saying. It fires and afterwards I think about it and wonder if it really is bogus.
If someone tells me that love isn't rational it is very likely that their communication style is going to ping, "That's bullshit." A...
Entire ways of acting and reacting - even mini-facets of personalities, tones of voice, turns of phrase - are also cached. These don't have to be from someone else - they can be from the "you" of 10 years ago (which may have been a composite of your role models at the time). They are otherwise known as habits that you haven't updated or re-evaluated for a long time.
The mini-pattern of action worked (or made sense) when you were 7, and it's so second-nature that it hasn't even entered your conscious awareness since then to give you a chance to reassess it.
Who needs a meme or a cached thought to ask. Can you ask one question at a time or make less than one statement without so many words?
Interesting article, but I'm not so sure about the "cache" analogy. A typical cache in computer science has two major differences with the effect you're pointing to :
A cache stores the result of a computation. Result of a complex algorithm, of a database of external server query, of disk read, ... but the computation is done once and then the result is stored for later used. Very few cache in computer science are caching results that comes from elsewhere but that were not computed at least once. While in your case, it's not "I did once th
I believe Schopenhauer came to the same conclusion.
"Reading is merely a surrogate for thinking for yourself; it means letting someone else direct your thoughts. Many books, moreover, serve merely to show how many ways there are of being wrong, and how far astray you yourself would go if you followed their guidance. You should read only when your own thoughts dry up, which will of course happen frequently enough even to the best heads; but to banish your own thoughts so as to take up a book is a sin against the holy ghost; it is like deserting untramm...
"One neuropsychologist estimates that visual perception is 90 percent memory, less than 10 percent sensory [nerve signals]." Apparently, we even use cached thought to see. We're really biased, huh?
An example of a cached thought reported by Alex Blumberg in This American Life, episode 293: "A Little Bit of Knowledge."
...I can reconstruct the events that led me to one of the most embarrassing conversations of my adult life. The chain starts back when I was 11 or 12, and I first heard the term Nielsen family. I was probably listening to some adults talk. And from their conversation I gathered that networks consulted Nielsen families to find out how popular a television show was. But that didn't make sense. Why would they only ask people named Nielsen which shows they liked. I started thinking.
I knew that when they figured things like this out, they didn't ask everybody, they just asked a small percentage of people, and then extrapolated. I think I figured they had done some research and found that the name Nielsen-- because it was a common name maybe, and it seemed to cut across class and economic lines-- actually came pretty close to a representative sample. I knew this wasn't the way they measured public opinion now, but it seemed like the Nielsen surveys had been around for a while. And I figured they were just a holdover from a more primitive, less statistically ri
From experience I find that the appeal to nature fallacy dominates cached thoughts manifesting itself mainly into conservatism. For example when I broached the topics of life extension with my mother.
From A New Kind of Science by Stephen Wolfram, page 0621:
Usually a small amount of thinking allows us to identify at least some regularities. But typically these regularities are ones that can also be found quite easily by many of the standard methods of perception and analysis discussed earlier in this chapter.
So what then does human thinking in the end have to contribute? The most obvious way in which it stands out from other methods of perception and analysis is in its large-scale use of memory.
one trick you'd use as heavily as possible...
Does that mean there are other usable tricks?
I was curious so I looked up the reasoning (and original paper) behind the hundred-step rule.
"Connectionist Models and Their Properties" (http://csjarchive.cogsci.rpi.edu/1982v06/i03/p0205p0254/MAIN.PDF)
"Neurons whose basic computational speed is a few milliseconds must be made to account for complex behaviors which are carried out in a few hundred milliseconds (Posner, 1978). This means that entire complex behaviors are carried out in less than a hundred time steps."
How do you consider interpretation of the cache? For example,
"Death gives rise to meaning" can be interpreted in many different ways, as some can see it as inspiring while others as meaningless, confusing or untrue.
But as you said, we can't actually recompute everything. No time. So the exhortation to "think!" can't possibly be followed in more than a small fraction of the cases.
The best we can do is to occasionally recompute certain items. And, if the re-computation is significantly at odds with the cached result, communicate this to others, who are likely to have the same cached result. We can do this in parallel. You can recompute a few things, I'll recompute a few things, and thousands of others are meanwhile recomputing a few things. Occasionally someone may have a significantly different result, which he'll hopefully communicate to others. The number of significantly different results will hopefully be only a small fraction of the number of recomputed results, which might bring the sharing of different results within the realm of possibility. For example, if there are 100 of us, and each recomputes 10 results, then collectively we recompute 1000 results (assuming no overlap). Only 1 out of every 100 recomputed results might be different from the cached result. So we only need to share among ourselves the 10 significantly different recomputed results. That is pretty easy, and we will in effect have done an overhaul of 1000 cached results, at the price of only 10 recomputations each and 10 received communications each (and one transmitted communication for each person who computed a new result). Seems doable.
Two problems.
First, each of us has a different mind that produces a different thought cache, and most of us probably won't be able to find much of a trunk build that we can agree on. To avoid conflicts, we'll have to transition from the current monolithic architecture to a Unix-like modular architecture. But that will take years, because we'll have to figure out who's running what modules, and which modules each entry in the thought cache comes from. (You can't count on lsmod to give complete or accurate results. I'd been running several unnamed modules fo...
One of the single greatest puzzles about the human brain is how the damn thing works at all when most neurons fire 10–20 times per second, or 200Hz tops. In neurology, the “hundred-step rule” is that any postulated operation has to complete in at most 100 sequential steps—you can be as parallel as you like, but you can’t postulate more than 100 (preferably fewer) neural spikes one after the other.
Can you imagine having to program using 100Hz CPUs, no matter how many of them you had? You’d also need a hundred billion processors just to get anything done in realtime.
If you did need to write realtime programs for a hundred billion 100Hz processors, one trick you’d use as heavily as possible is caching. That’s when you store the results of previous operations and look them up next time, instead of recomputing them from scratch. And it’s a very neural idiom—recognition, association, completing the pattern.
It’s a good guess that the actual majority of human cognition consists of cache lookups.
This thought does tend to go through my mind at certain times.
There was a wonderfully illustrative story which I thought I had bookmarked, but couldn’t re-find: it was the story of a man whose know-it-all neighbor had once claimed in passing that the best way to remove a chimney from your house was to knock out the fireplace, wait for the bricks to drop down one level, knock out those bricks, and repeat until the chimney was gone. Years later, when the man wanted to remove his own chimney, this cached thought was lurking, waiting to pounce . . .
As the man noted afterward—you can guess it didn’t go well—his neighbor was not particularly knowledgeable in these matters, not a trusted source. If he’d questioned the idea, he probably would have realized it was a poor one. Some cache hits we’d be better off recomputing. But the brain completes the pattern automatically—and if you don’t consciously realize the pattern needs correction, you’ll be left with a completed pattern.
I suspect that if the thought had occurred to the man himself—if he’d personally had this bright idea for how to remove a chimney—he would have examined the idea more critically. But if someone else has already thought an idea through, you can save on computing power by caching their conclusion—right?
In modern civilization particularly, no one can think fast enough to think their own thoughts. If I’d been abandoned in the woods as an infant, raised by wolves or silent robots, I would scarcely be recognizable as human. No one can think fast enough to recapitulate the wisdom of a hunter-gatherer tribe in one lifetime, starting from scratch. As for the wisdom of a literate civilization, forget it.
But the flip side of this is that I continually see people who aspire to critical thinking, repeating back cached thoughts which were not invented by critical thinkers.
A good example is the skeptic who concedes, “Well, you can’t prove or disprove a religion by factual evidence.” As I have pointed out elsewhere,1 this is simply false as probability theory. And it is also simply false relative to the real psychology of religion—a few centuries ago, saying this would have gotten you burned at the stake. A mother whose daughter has cancer prays, “God, please heal my daughter,” not, “Dear God, I know that religions are not allowed to have any falsifiable consequences, which means that you can’t possibly heal my daughter, so . . . well, basically, I’m praying to make myself feel better, instead of doing something that could actually help my daughter.”
But people read “You can’t prove or disprove a religion by factual evidence,” and then, the next time they see a piece of evidence disproving a religion, their brain completes the pattern. Even some atheists repeat this absurdity without hesitation. If they’d thought of the idea themselves, rather than hearing it from someone else, they would have been more skeptical.
Death. Complete the pattern: “Death gives meaning to life.”
It’s frustrating, talking to good and decent folk—people who would never in a thousand years spontaneously think of wiping out the human species—raising the topic of existential risk, and hearing them say, “Well, maybe the human species doesn’t deserve to survive.” They would never in a thousand years shoot their own child, who is a part of the human species, but the brain completes the pattern.
What patterns are being completed, inside your mind, that you never chose to be there?
Rationality. Complete the pattern: “Love isn’t rational.”
If this idea had suddenly occurred to you personally, as an entirely new thought, how would you examine it critically? I know what I would say, but what would you? It can be hard to see with fresh eyes. Try to keep your mind from completing the pattern in the standard, unsurprising, already-known way. It may be that there is no better answer than the standard one, but you can’t think about the answer until you can stop your brain from filling in the answer automatically.
Now that you’ve read this, the next time you hear someone unhesitatingly repeating a meme you think is silly or false, you’ll think, “Cached thoughts.” My belief is now there in your mind, waiting to complete the pattern. But is it true? Don’t let your mind complete the pattern! Think!
1See ‘Religion’s Claim to be Non-Disprovable,” in Map and Territory.