Except the universe doesn't care how much backbreaking effort you make, only if you get the cause and effect right. Which is why cultures that emphasize hard work are not overtaking cultures that emphasize reason (Enlightenment cultures). Of course even these cultures must still do some work, that of enacting their cleverly thought out causes.
This doesn't seem to mesh with the Friendly AI goal of getting it perfectly right on the first try.
Do we accept some uncertainty and risk to do something extraordinary now, or do we take the slow, calm, deliberative course that stands a chance of achieving perfection?
Is there any chance of becoming a master of the blade without beginning to cut?
This can be solved with an expected utility calculation.
Or to put it another way: if messing up with a sword even once meant that you would almost certainly lop your leg off, then it would make a whole lot of sense to use only a Nerf sword until you became a master.
Ian C: Rome emphasized hard work and discipline, Greece reason. Arguably this applies to China and Denmark too. The dominant enlightenment cultures, such as Britain in 1750 or 1900, weren't merely better at reason. They were ABSOLUTELY CRUSHINGLY better at reason AND hard work than were other world cultures at the time. Modern developed cultures are clearly in the middle ground between the enlightenment pre-WWI and the the ancient third world.
How long would it take for an AGI engineer from the (seemingly inevitable) future to explain it to you so you could build one? Is it humanly possible - is a single person capable of understanding and remembering all of it? It's been said there's not one person who understands a modern microcomputer in its totality - perhaps the last of the low hanging fruit was picked by people in the age of Steve Wozniak.
But, on the other hand, this sort of thing probably isn't a good idea. Or at least there should be a more compelling reason for it.
Extraordinary results require extraordinary effort? Not necessarily. You may achieve extraordinary results, effortlessly like water flowing around rocks. Don't consider the problem domain as outside of yourself. Become a part of the problem system, so that it effortlessly self-optimizes and develops as if by itself and you merely find yourself in the midst of it every day, feeling more like an observer than a participator. When it feels like an effort, you're not maximally tuned to the problem.
By being maximally tuned to the problem, I mean that you have maximized your knowledge about the domain (you know the dead-ends) and your techniques and methods are sufficient in number and capability (your know all the tricks and have created some of your own); you lack nothing and can simply flow.
> "You should go to college and get a Master's degree and get a doctorate and publish a lot of papers on ordinary things -
scientists and investors won't listen to you otherwise." Even assuming that I tested out of the bachelor's degree... You can "test out of" every step to the ladder, and go straight to post-doc/professor equivalent by publishing multiple well-respected papers (and not necessarily on ordinary things) in top journals.
Small Typo Alert: The second quote should be attributed to "Mastering Eishin-Ryu Swordsmanship"
"Ryu", not "Ruy".
I'm all for putting effort into the little extraordinary, we just have different definitions of extraordinary.
A little extraordinary is building a human type brain stuff. By type I mean limited in its self-understanding due to complex hacked together constantly changing code. Then spreading it to enough people so that any one persons respective mental problems get evened out so the society thus created is not dominated by any one group. Convincing the society that we might need to relax privacy might before we get super plagues, is also a little extraordinary.
Very extraordinary is building something that understands itself and the laws of physics enough to be able to expand itself exponentially taking over nearby space uploading all of humanity into a simulation where death was not allowed. And getting it right first time.
It is not sadness in the future that means I do not want to be there, it is perpetual responsibility. I only want temporary responsibility, because it is mentally draining, however I do want responsibility while I am alive.
Most American culture seems like a reinvention of British culture demanded by national pride. My impression is that their versions are like cheap knock-offs of the originals. Their beers are worse. They even managed to mess up the game of football. America faces limitations due to their vast tracts of underpopulated flyover country. That's a problem Britain doesn't have.
In my experience, 'isshokenmei' is a rote expression drilled in Japanese schoolkids from nursery school on. As such, it is long since drained of any "deep" meaning to your average Japanese. I'm not sure a parent exhorting a Japanese kid to do well on a test with "isshokenmei" is saying much more than "Try your best."
You are absolutely right about Japanese science not being pre-eminent in the world and why. For related reasons I am leaving the East altogether - the action is in the US (and the West in general), and looks to stay there for a long time.
And Tim Tyler, viewing Japanese culture as a shabby simulacrum of Chinese culture makes you sound like an executive at GM or Ford from the 70s talking about Toyota or Honda. Who's laughing now? Sure Japanese culture was heavily, heavily influenced in various waves by China - that is, pre 19th-century China, going back to the 6th and 7th centuries. I would say today foreign influence on Japan emanates almost exclusively from the West, particularly the US (maybe a little pop culture from Korea). If anything, these days China is copying Japan.
...expand itself exponentially taking over nearby space uploading all of humanity into a simulation
Ah yes, there's nothing wrong with murdering people as long as you name video game characters after them.
If you had every cell in your body replaced individually with synthesized cells with identical properties, you would retain your memories, identity, and supposedly your existence. After all, most of your body has been replaced several times already.
How is it different if you take your thoughts and swap them from a human body to a computer? Are you really dead, or just in a glorified full body prosthetic?
One aspect of present in most of the Japanese arts is repetition and training. In order to achieve excellence at something, you do it over and over again hundreds and thousands of times until you get it right.
In activities where constant repetition is possible and productive, reliable progress can be achieved. Training in this way dominates even intelligence. An average IQ chess player, karateka, language student, or tennis player who has trained for thousands of hours will be far more effective at the activity than a genius level beginner.
One problem with rationality is it is hard to practice. If you want to train for a marathon, you get up at 6am every morning and run ten miles. Is there an analogous training method for rationality? If so, it is not obvious to me. This lack of an effective training method is probably why intelligence appears to be the dominant factor in rationality contests.
Sometimes I do wonder if I ought to be peddling rationality in Japan instead of the United States - but Japan is not preeminent over the United States scientifically, despite their more studious students. The Japanese don't rule the world today, though in the 1980s it was widely suspected that they would (hence the Japanese asset bubble). Why not? In the West, there is a saying: "The squeaky wheel gets the grease." In Japan, the corresponding saying runs: "The nail that sticks up gets hammered down." This is hardly an original observation on my part: but entrepreneurship, risk-taking, leaving the herd, are still advantages the West has over the East. And since Japanese scientists are not yet preeminent over American ones, this would seem to count for at least as much as desperate efforts.
The Japanese don't rule the world today, though in the 1980s it was widely suspected that they would True, however, they didn't blow up their entire financial system and in turn that, seemingly, of the entire planet.
And since Japanese scientists are not yet preeminent over American ones, this would seem to count for at least as much as desperate efforts. http://www.crunchgear.com/2008/08/01/the-us-beats-japan-in-patent-application-filing-for-the-first-time-since-1963-becomes-world-no-1/ http://www.seinan-gu.ac.jp/~djohnson/natural/population.html
Maybe you and I do math differently, but based on population, Japan is cleaning America's clock, assuming patents are a reasonable indicator of innovation.
Any reasonable person, even watching the mainstream media news, could hardly come to any other conclusion than America has backed itself into a corner. It's pretty simple math.
Make no mistake, I understand your point, that America is more dynamic and all that, but as it is, they have set things up almost perfectly to destroy themselves (financially), and the entire world if they get away with it. Perhaps if America had adopted a little more of the "The nail that sticks up gets hammered down" philosophy, we wouldn't be in this predicament.
[...]assuming patents are a reasonable indicator of innovation.
Ye hairy tentacled gods but why would anyone assume that?
It is an indicator that you would hope would show correlation. Unfortunately, there are people who will just patent every single idea they have regardless of merit. Still, since some level of uniqueness and practicality is required for a patent, the people who can make the most patentable ideas can probably also generate useful additions to the model of rationality the fastest.
On the other hand, since Eliezer is trying to spread rationality, it makes sense to target the largest population that can benefit, using Japanese would drastically reduce the number who can read his works. Even if you could show that the Japanese were on average slightly more rational to begin with, you would end up with less people who learned all he could teach them.
Yes, agreed on both counts. Though, I think there are also other big issues with using patents as an indicator of innovation:
Whether or not you should get a patent for a given innovation is a cultural and situational issue. For example, an Open Source software developer is unlikely to seek a patent for their work, as is a university researcher who is hoping to publish their results. Using patents as an indicator might be messed up by the two cultures having different emphases on styles and methods of innovation.
A patent contains a variable amount of innovation: depending on how you want to package it, a given idea might be encapsulated in one big patent or in ten small ones. So, using patents as an indicator might be messed up by the two cultures tending to, for whatever practical or sociological or economic reason, group up or split up their patent applications.
Finally, this would be the easiest thing to research, but I'm not sure if the US patent office and the Japanese patent office use the same thresholds for minimum innovation, de facto.
Yeah, all three of those points make using patents to judge Innovation almost useless. Until you compare the cultural differences and patent requirements more, it still counts as weak evidence though.
I think most people's feedback threshold requires some return on their efforts in a relatively short time period. It takes monk-like patience to work on something indefinitely without any intermediary returns. So then, I don't think the point in contention is whether people are willing to make extraordinary effort, it is whether they are willing to make extraordinary effort without extraordinary returns in a time span relative to their feedback threshold. Even in eastern cultures where many people believe that enlightenment in the strong sense is possible by meditating your whole life, there is a reason why there are only a few practicing monks.
Most American culture seems like a reinvention of British culture demanded by national pride. My impression is that their versions are like cheap knock-offs of the originals. Their beers are worse. They even managed to mess up the game of football.
Though that is apparently meant sardonically, as a UK citizen, it doesn't seem too far off to me - though I would have gone for the language and the arts with my comparison.
Of course the analogy goes horribly wrong when you get down to the "numbers" point I mentioned. Japan is tiny, China is huge. England is tiny, America is huge. The very same factor that worked for America over England also works for China over Japan. Almost everyone realises this.
True, however, they didn't blow up their entire financial system and in turn that, seemingly, of the entire planet.
I'm not sure if this should be taken literally, but see wiki:Japanese asset price bubble and wiki:Carry trade#Currency.
In activities where constant repetition is possible and productive, reliable progress can be achieved. Training in this way dominates even intelligence. An average IQ chess player, karateka, language student, or tennis player who has trained for thousands of hours will be far more effective at the activity than a genius level beginner. One problem with rationality is it is hard to practice. If you want to train for a marathon, you get up at 6am every morning and run ten miles. Is there an analogous training method for rationality? If so, it is not obvious to me. This lack of an effective training method is probably why intelligence appears to be the dominant factor in rationality contests.
I also complained about the lack of a practice method in "The Martial Art of Rationality", my very first post here on Overcoming Bias.
But your suggestion that this is why rationality-contests are dominated by intelligence is very, very deep. I'm not sure how true it is (a little true or a whole lot true) but it's definitely truly deep, in the good sense of the term.
(What else could we generalize this to?)
Along the lines of a rationality competition and why rationalists don't seem to do all that much better, is there a criteria for rationality? In the fields where lots of work results in maximizing gain, there are quantitative criteria for what counts as good and bad. I don't know of any such criteria for rationality.
Right on schedule: "Two Americans and a U.S.-based Japanese scientist won the Nobel Prize in chemistry on Wednesday" ... Here.
Also, as per Daniel Burfoot's comment, the Japanese have a saying that you don't truly know how to do something until you've done it 10,000 times. The goal of that as I understand it, in martial arts or language training or any other repetitive art, is to go beyond thought, hence, beyond intelligence, to the place where you are engaging in pure action.
Is it possible to practice rationality in such a reflexive manner? Probably not. Thus the "fallback" on intelligence in rationality contests.
Hmm, it would be interesting to try to develop a video game aimed at teaching rationality; I'm sure it would be possible to make a game world with the kind of baffling chaotic problem Eliezer talks about. It could be presented as a world with weird physical laws, in which you perform experiements ... but it could also be something completely different, as long as the baffling element is there.
There are some aspects that are hard to reproduce in a game, such as the fact that you don't even know whether a solution exists and/or how much effort is required, or the social pressure to do things a particular way ... but then there are plenty of ways game have of teaching you things that are subtly wrong...
Anyway, even if it's not possible to make a game that makes you really challenge a baffling chaotic problem, it's probably still possible to make one that can teach you quite a bit about rationality.
(by the way, I'm a game programmer / game designer, so I'll start taking notes on what could be possible, what's already been done etc.) (I already know of a game that gave you "incorrect" feedback on the rules - "don't do this or you'll die" where in fact, you don't always die if you don't)
Eliezer: I also complained about the lack of a practice method in "The Martial Art of Rationality", my very first post here on Overcoming Bias.
But your suggestion that this is why rationality-contests are dominated by intelligence is very, very deep.
Surely it is a tautology. "Intelligence" is what we say people have, when they do well at rationality contests. Compare "talent", which is what we say people have, when they exhibit prowess at a practisable activity beyond what would be expected from the practise they have put in.
Re: why rationality can't be learned by rote -
If you introspect on a process of reason, you see that you actually choose at each step which path of inquiry to follow next and which to ignore. Each choice takes the argument to the next step, ultimately driving it to completion. Reason is "powered by choice(TM)" which is why it is incoherent to argue rationally for determinism and also why it can't be learned by rote.
Software developers (such as myself) in our more abstract moments can think of reason as simply encoding ones premises as a string of symbols standing for definitions and mechanically applying the rules of deduction (Prolog style). But introspection belies this - it's actually highly creative and messy. Reason is an art not a science.
...the language and the arts with my comparison...
I thought about going this way, but I decided to stick with what I know.
Since sarcasm seems to have failed, let me just state flatly that all of the cultures we've mentioned have enough members and enough diversity that blanket assertions such as, "Japanese martial arts are worse than Chinese ones," or "American football is a cheap knockoff of rugby" are reductive and parochial to the point of not-even-wrongness.
One last wrinkle:
I think that it's important to understand the value of selectivity. One simply doesn't have the energy and time to make an extraordinary effort at all aspects of one's life.
Pick a few areas in which to make an extraordinary effort, and focus on getting by everywhere else.
My Japanese is rudimentary at best, but isn't issho related to issho ni meaning "together"? And what about the practice of saying gambatte kudasai before exams -- "please do your best"? Maybe the Japanese have as many ways of describing effort as they do to describe the circumstances of death.
Make no mistake, I understand your point, that America is more dynamic and all that, but as it is, they have set things up almost perfectly to destroy themselves (financially), and the entire world if they get away with it. Perhaps if America had adopted a little more of the "The nail that sticks up gets hammered down" philosophy, we wouldn't be in this predicament.Iceland. Home of conformity and Tall Poppy Syndrome. Their entire country is in massive debt.
Maybe making poor economic decisions has nothing to do with ideology or cultural assumptions...
Or maybe there are a kajillion variables and we need more data points before we can make good hypotheses.
Aaaand the takeaway metaphor is that 'creative' ideas are the probably explosive ones, but we sometimes still really need to move trucks.
Solidly great, this.
"It is essential for a man to strive with all his heart, and to understand that it is difficult even to reach the average if he >does not have the intention of surpassing others in whatever he does." —Budo Shoshinshu
seems to contradict the 'give up already' sequence
Every now and then, someone asks why the people who call themselves "rationalists" don't always seem to do all that much better in life
I think this is because, while we have lots of useful theory and advice here about epistemic rationalism, we have virtually nothing about instrumental rationalism.
1. Success requires taking action, and taking action requires generating options, and I'm not very good at generating options. Little in the way of advice for doing it is given in Rationality A-Z (at least up to this point) beyond "spend a few minutes thinking about it, if it's important." It's good advice, but seems insufficient. (I would kind of expect a leading expert in AGIs to have more ideas about this problem—or are the techniques used by AIs inapplicable to humans?)
2. Motivation is perhaps a larger problem for me. I don't know how motivation works and how to create it in myself; in theory there is a thing I want to create more than anything else in the world; in practice I just haven't felt like doing it for several weeks (perhaps this is because I anticipate no one will see its utility and almost everyone will ignore it as they have in the past, but actually the rationale lies in my emotional system, it cannot be introspected, and sometimes my feelings change so I can work on it again.) "Shut up and do the impossible!", says the next post, but this requires quite some motivation.
3. A lot of success in life depends on our relationships with others, and I've never been good at developing relationships, nor has there been much advice about that here. I don't know any aspiring rationalists in person, and I find the poor reasoning of most people to be grating. I can't use terms like "expected value" with others and expect to be understood. Succeeding at standard non-rationalist office politics is one of those things that I'd love to do in theory, but in practice it's unpredictable and mysterious and scary and I lack the will to take the necessary risks (especially having lost my last two jobs, I really want something stable at the moment). I might worry that I'll never be more than a low-level employee, if it would do any good to worry. I often lament that I play the role of a "leaf node" in the game of life—a person no one pays much attention to—but I simply don't know how to fix the problem.
Bayesians with the same priors cannot agree to disagree, is a believing Orthodox Jew.
The facile explanation is that people compartmentalize and have biases, but this reminds me, where do priors come from? So far I have not seen any proposals for how to evaluate evidence in a new area of study, let alone how to evaluate evidence "from scratch".
Or the ones who say to me: "You should go to college and get a Master's degree and get a doctorate and publish a lot of papers on ordinary things—scientists and investors won't listen to you otherwise." Even assuming that I tested out of the bachelor's degree, we're talking about at least a ten-year detour in order to do everything the ordinary, normal, default way. And I stand there thinking: Are they really under the impression that humanity can survive if every single person does everything the ordinary, normal, default way?
I guess it's not exactly a coincidence, but I had to make the near same judgment not two weeks ago.
Robert Aumann, the Nobel laureate who first proved that Bayesians with the same priors cannot agree to disagree, is a believing Orthodox Jew. Surely he understands the math of probability theory, but that is not enough to save him. What more does it take?
This reminds me of buddhist meditation: understanding the path intellectually may be useful, but what really matters is living it.
I think this applies to any kind of knowledge ; it's fine to know something, but not so useful if we don't act on it. We could even go as far as saying that we don't truly know something until we act on it, for example on the premise that we want to be healthy and smart and sport is a cost-effective way to improve on these, then if we are not practicing sport clearly our brain doesn't understand deeply enough the premise, because if it did the neurons would we wired in a way that would make us practice. So that's what true understanding might be: having neurons wired in a way that make us do the thing.
Unfortunately my experience of school is that it doesn't teach kids to act on what they know, maybe it assumes it happens auto-magically or maybe it doesn't care, but it seems to like a blind spot.
"It is essential for a man to strive with all his heart, and to understand that it is difficult even to reach the average if he does not have the intention of surpassing others in whatever he does."
—Budo Shoshinshu"In important matters, a 'strong' effort usually results in only mediocre results. Whenever we are attempting anything truly worthwhile our effort must be as if our life is at stake, just as if we were under a physical attack! It is this extraordinary effort—an effort that drives us beyond what we thought we were capable of—that ensures victory in battle and success in life's endeavors."
—Flashing Steel: Mastering Eishin-Ryu Swordsmanship
"A 'strong' effort usually results in only mediocre results"—I have seen this over and over again. The slightest effort suffices to convince ourselves that we have done our best.
There is a level beyond the virtue of tsuyoku naritai ("I want to become stronger"). Isshoukenmei was originally the loyalty that a samurai offered in return for his position, containing characters for "life" and "land". The term evolved to mean "make a desperate effort": Try your hardest, your utmost, as if your life were at stake. It was part of the gestalt of bushido, which was not reserved only for fighting. I've run across variant forms issho kenmei and isshou kenmei; one source indicates that the former indicates an all-out effort on some single point, whereas the latter indicates a lifelong effort.
I try not to praise the East too much, because there's a tremendous selectivity in which parts of Eastern culture the West gets to hear about. But on some points, at least, Japan's culture scores higher than America's. Having a handy compact phrase for "make a desperate all-out effort as if your own life were at stake" is one of those points. It's the sort of thing a Japanese parent might say to a student before exams—but don't think it's cheap hypocrisy, like it would be if an American parent made the same statement. They take exams very seriously in Japan.
Every now and then, someone asks why the people who call themselves "rationalists" don't always seem to do all that much better in life, and from my own history the answer seems straightforward: It takes a tremendous amount of rationality before you stop making stupid damn mistakes.
As I've mentioned a couple of times before: Robert Aumann, the Nobel laureate who first proved that Bayesians with the same priors cannot agree to disagree, is a believing Orthodox Jew. Surely he understands the math of probability theory, but that is not enough to save him. What more does it take? Studying heuristics and biases? Social psychology? Evolutionary psychology? Yes, but also it takes isshoukenmei, a desperate effort to be rational—to rise above the level of Robert Aumann.
Sometimes I do wonder if I ought to be peddling rationality in Japan instead of the United States—but Japan is not preeminent over the United States scientifically, despite their more studious students. The Japanese don't rule the world today, though in the 1980s it was widely suspected that they would (hence the Japanese asset bubble). Why not?
In the West, there is a saying: "The squeaky wheel gets the grease."
In Japan, the corresponding saying runs: "The nail that sticks up gets hammered down."
This is hardly an original observation on my part: but entrepreneurship, risk-taking, leaving the herd, are still advantages the West has over the East. And since Japanese scientists are not yet preeminent over American ones, this would seem to count for at least as much as desperate efforts.
Anyone who can muster their willpower for thirty seconds, can make a desperate effort to lift more weight than they usually could. But what if the weight that needs lifting is a truck? Then desperate efforts won't suffice; you'll have to do something out of the ordinary to succeed. You may have to do something that you weren't taught to do in school. Something that others aren't expecting you to do, and might not understand. You may have to go outside your comfortable routine, take on difficulties you don't have an existing mental program for handling, and bypass the System.
This is not included in isshokenmei, or Japan would be a very different place.
So then let us distinguish between the virtues "make a desperate effort" and "make an extraordinary effort".
And I will even say: The second virtue is higher than the first.
The second virtue is also more dangerous. If you put forth a desperate effort to lift a heavy weight, using all your strength without restraint, you may tear a muscle. Injure yourself, even permanently. But if a creative idea goes wrong, you could blow up the truck and any number of innocent bystanders. Think of the difference between a businessman making a desperate effort to generate profits, because otherwise he must go bankrupt; versus a businessman who goes to extraordinary lengths to profit, in order to conceal an embezzlement that could send him to prison. Going outside the system isn't always a good thing.
A friend of my little brother's once came over to my parents' house, and wanted to play a game—I entirely forget which one, except that it had complex but well-designed rules. The friend wanted to change the rules, not for any particular reason, but on the general principle that playing by the ordinary rules of anything was too boring. I said to him: "Don't violate rules for the sake of violating them. If you break the rules only when you have an overwhelmingly good reason to do so, you will have more than enough trouble to last you the rest of your life."
Even so, I think that we could do with more appreciation of the virtue "make an extraordinary effort". I've lost count of how many people have said to me something like: "It's futile to work on Friendly AI, because the first AIs will be built by powerful corporations and they will only care about maximizing profits." "It's futile to work on Friendly AI, the first AIs will be built by the military as weapons." And I'm standing there thinking: Does it even occur to them that this might be a time to try for something other than the default outcome? They and I have different basic assumptions about how this whole AI thing works, to be sure; but if I believed what they believed, I wouldn't be shrugging and going on my way.
Or the ones who say to me: "You should go to college and get a Master's degree and get a doctorate and publish a lot of papers on ordinary things—scientists and investors won't listen to you otherwise." Even assuming that I tested out of the bachelor's degree, we're talking about at least a ten-year detour in order to do everything the ordinary, normal, default way. And I stand there thinking: Are they really under the impression that humanity can survive if every single person does everything the ordinary, normal, default way?
I am not fool enough to make plans that depend on a majority of the people, or even 10% of the people, being willing to think or act outside their comfort zone. That's why I tend to think in terms of the privately funded "brain in a box in a basement" model. Getting that private funding does require a tiny fraction of humanity's six billions to spend more than five seconds thinking about a non-prepackaged question. As challenges posed by Nature go, this seems to have a kind of awful justice to it—that the life or death of the human species depends on whether we can put forth a few people who can do things that are at least a little extraordinary. The penalty for failure is disproportionate, but that's still better than most challenges of Nature, which have no justice at all. Really, among the six billion of us, there ought to be at least a few who can think outside their comfort zone at least some of the time.
Leaving aside the details of that debate, I am still stunned by how often a single element of the extraordinary is unquestioningly taken as an absolute and unpassable obstacle.
Yes, "keep it ordinary as much as possible" can be a useful heuristic. Yes, the risks accumulate. But sometimes you have to go to that trouble. You should have a sense of the risk of the extraordinary, but also a sense of the cost of ordinariness: it isn't always something you can afford to lose.
Many people imagine some future that won't be much fun—and it doesn't even seem to occur to them to try and change it. Or they're satisfied with futures that seem to me to have a tinge of sadness, of loss, and they don't even seem to ask if we could do better—because that sadness seems like an ordinary outcome to them.
As a smiling man once said, "It's all part of the plan."