A Call for Constant Vigilance

katydee

Related to: What Do We Mean By "Rationality?"

Rationality has many facets, both relatively simple and quite complex. As a result, it can often be hard to determine what aspects of rationality you should or shouldn't stress.

An extremely basic and abstract model of how rationality works might look a little something like this:

Collect evidence about your environment from various sources
Update your model of reality based on evidence collected (optimizing the updating process is more or less what we know as epistemic rationality)
Act in accordance with what your model of reality indicates is best for achieving your goals (optimizing the actions you take is more or less what we know as instrumental rationality)
Repeat continually forever

A lot of thought, both on LessWrong and within the academic literature on heuristics and biases, has gone into improving epistemic rationality, and while improving instrumental rationality was less of a focus at first, recently the community has been focusing more on it. On the other hand, improving your ability to collect evidence has been relatively neglected-- hence the (in-progress as of this writing) Situational Awareness sequence.

But most neglected of all has been the last step, "repeat continually forever." This sounds like a trivial instruction but is in fact highly important to emphasize. All your skills and training and techniques mean nothing if you don't use them, and unfortunately there are many reasons that you might not use your skills.

You might be offended, angry, hurt, or otherwise emotionally compromised. Similarly, you might be sleepy, inebriated, hungry, or otherwise physically compromised. You might be overconfident in your ability to handle a certain type of problem or situation, and hence not bother to think of other ways that might work better.^[1] You might simply not bother to apply your skills because you don't think they're necessary, missing out on potential gains that you don't see at a glance-- or maybe even don't know exist. All in all, there are many times in which you may be missing out on the benefits that your skills can provide.

It may therefore be worthwhile to occasionally check whether or not you are actually applying your skills. Further, try to make this sort of check a habit, especially when encountering circumstances where people would typically be less than rational. If you find that you aren't using your skills as often as you'd expect, that may be cause for alarm, and at the very least is cause for introspection. After all, if rationality skills can be constantly applied to being successful in everyday life, we should be constantly on the watch for opportunities to apply them, as well as for potential lapses in our vigilance.

I indeed suspect that most LessWrong users would benefit more from being more vigilant in practicing and applying basic rationality skills than they would from learning cool advanced techniques.This principle is generally true in the martial arts, and both the inside and outside view strongly suggest to me that it is true for the art of rationality as well.

All in all, improving your rationality is a matter of serious practice and changing your mindset, not just learning cool new life hacks-- so next time you think about improving your rationality, don't look for new tricks, but new ways to truly integrate the principles you are already familiar with.

[1] The footnote to Humans Are Not Automatically Strategic describes several examples where this might apply.

Related to: What Do We Mean By "Rationality?"

Rationality has many facets, both relatively simple and quite complex. As a result, it can often be hard to determine what aspects of rationality you should or shouldn't stress.

An extremely basic and abstract model of how rationality works might look a little something like this:

Collect evidence about your environment from various sources
Update your model of reality based on evidence collected (optimizing the updating process is more or less what we know as epistemic rationality)
Act in accordance with what your model of reality indicates is best for achieving your goals (optimizing the actions you take is more or less what we know as instrumental rationality)
Repeat continually forever

[1] The footnote to Humans Are Not Automatically Strategic describes several examples where this might apply.

My understanding of the "quantity to quality conversion" phrase is that in many situations the relation between some inputs and outputs is not linear. More specifically, there are many situations where at the beginning the relation seems linear, but later at some point the increase of outputs becomes incredibly huge (incredibly = for people who based their models on extrapolating the linear relationship at the beginning). Even more specifically, you can have one input "A" that has obvious effect on "X", but almost zero effect on "Y" and "Z". Then at some moment with additional increases of "A" also "Y" and "Z" start growing (which was totally unexpected by the old model).

Specific example: You start playing piano. At the beginning, it feels like it has a simple linear impact on your life. You spend 1 hour playing piano, you get an ability to play a simple song quite well. You spend 2 hours playing piano, you get an ability to play another simple song quite well. Extrapolate this, and you get a model. According to this model, after spending 80000 hours playing piano, you would expect to be able to play 80000 simple songs quite well. -- What happens in reality is that you get an ability to play any simple song well just by looking at the music sheets, an ability to play very complex music, an ability to make money by playing the music, you become famous, get a lot of social capital, lot of friends, lot of sex, lot of drugs, etc. (Both non-linear outputs, and the outputs not predicted by the original model.)

A similar pattern appears in many different situations, so some people invented a mysteriously sounding phrase to describe it. Now it seems like some law of nature. But maybe it is just a selection effect (some situations develop like this, and we notice "oh, the law of quantity to quality conversion", other situations don't, and we ignore them).

In other words, "quantity" seems to mean "linear model", "quality" means "model", and the whole phrase decoded means "if you change variables enough, you may notice that the linear model does not reflect reality well (especially in situations where the curve starts growing slowly, and then it grows very fast)".

I was more after some discontinuity than a simple nonlinearity, like a quadratic or even an exponential dependence. And you are right, the selection effect is at work, but it's not a negative in this case. We want to select similar phenomena and find a common model for them, in order to be able to classify new phenomena as potentially leading to the same effects.

For example, if you look at some new hypothetical government policy which legislates indexing the minimum savings account rate to, say, inflation, you should be able to tell whether after a sizabl... (read more)

40

A Call for Constant Vigilance

40

40

40

A Call for Constant Vigilance

40

40