On the recent LessWrong/CFAR Census Survey, I hit the following question:

Which of the following major ethical systems do you subscribe to:

1) Consequentialism

2) Deontology

3) Virtue Ethics

4) Other

To my own surprise, I couldn't come up with a clear answer.  I certainly don't consistently apply one of these things across every decision I make in my life, and yet I consider myself at least mediocre on the scale of moral living, if not actually Neutral Good.  So what is it I'm actually doing, and how can I behave more ethical-rationally?

Well, to analyze my own cognitive algorithms, I do think I can actually place these various codes of ethics in relation to each other.  Basically, looked at behavioristically/algorithmically, they vary across how much predictive power I have, my knowledge of my own values, and what it is I'm actually trying to affect.

Consequentialism is the ethical algorithm I consider useful in situations of greatest predictive power and greatest knowledge of my own values.  It is, so to speak, the ethical-algorithmic ideal.  In such situations, the only drawback is that naive consequentialism fails to consider consequences on the person acting (ie: me).  Once I make that more virtue-ethical adjustment, consequentialism offers a complete ideal for ethical action over a complete spectrum of moral values for affecting both the universe and myself (but I repeat: I'm part of the universe).

However, in almost all real situations, I don't have perfect predictive knowledge -- not of the "external" universe and not of my own values.  In these situations, I can, however, use my incomplete and uncertain knowledge to find acceptable heuristics that I can expect to yield roughly monotonic behavior: follow those rules, and my actions will generally have positive effects.  This kind of thinking quickly yields up recognizable, regular moral commandments like, "You will not murder" or "You will not charge interest above this-or-that amount on loans".  Yes, of course we can come up with corner-case exceptions to those rules, and we can also elaborate logically on the rules to arrive at more detailed rules covering more circumstances.  However, by the time we've fully elaborated out the basic commandments into a complete, obsessively-compulsively detailed legal code (oh hello Talmud), we've already covered most of the major general cases of moral action.  We can now invent a criterion for how and when to transition from one level of ethical code to the one below it: our deontological heuristics should be detailed enough to handle any case where we lack the information (about consequences and values) to resort to consequentialism.

At first thought, virtue ethics seems like an even higher-level heuristic than deontological ethics.  The problem is that, unlike deontological and consequentialist ethics, it doesn't output courses of action to take, but instead short- and long-term states of mind or character that can be considered virtuous.  So we don't have the same thing here; it's not a higher-level heuristic but a seemingly completely different form of ethics.  I do think we can integrate it, however: virtue ethics just consists of a set of moral values over one's own character.  "What kind of person do I think is a good person?" might, by default, be a tautological question under strict consequentialism or deontology.  However, when we take an account of the imperfect nature of real people (we are part of the universe, after all), we can observe that virtue ethics serves as a convenient guide to heuristics for becoming the sort of person who can be relied upon to take right actions when moral issues present themselves.  Rather than simply saying, "Do the right thing no matter what" (an instruction that simply won't drive real human beings to actually do the right thing), virtue ethics encourages us to cultivate virtues, moral cognitive biases towards at least a deontological notion of right action.

It's also possible we might be able to separate virtue ethics into both heuristics over our own character, and actual values over our own character.  These two approaches to virtue ethics should then converge in the presence of perfect information: if I knew myself utterly, my heuristics for my own character would exactly match my values over my own character.

This is my first effort at actually blogging on rationality subjects, so I'm hoping it's not covering something hashed and rehashed, over and over again, in places like the Sequences, of which I certainly can't attest a full knowledge.

Under fairly weak assumptions, the most a standard rational economic agent is willing to pay for an item they don't own (WTP) and the least they're willing to accept in exchange for that item if they already own it (WTA) should be identical. In experiments with humans, psychologists and economists have repeatedly found WTP-WTA gaps suggesting that humans aren't rational in at least this specific way. This has been interpreted as the endowment effect* and evidence for prospect theory. According to prospect theory, people are loss averse. Roughly this means that that, given their current ownership set, people value not losing stuff more highly than gaining stuff. Thus once someone gains ownership of something they suddenly value it much more highly. This "endowment effect"* on one's valuation of an item has been put forth as an explanation for the observed WTP - WTA gaps.

*Wikipedia confusingly defines the endowment effect as the gap itself, i.e. as the phenomena to be explained instead of the explanation. I suspect this is a difference in terminology among economists and psychologists, where psychologists use the wiki definition and economists use the definition I give here. However, calling the WTP-WTA gap an "endowment effect" is a bit misleading because a priori the gap may not have anything to endowments at all.

A paper (pdf) by Charlie Plott and Kathryn Zeiler investigates WTP-WTA gaps and it turns out that they may just be due to subjects not quite understanding the experimental protocols, particularly in the value elicitation process. Here's an important quote from their conclusion, but do read the paper for details: 

The issue explored here is not whether a WTP-WTA gap can be observed. Clearly, the experiments of KKT and others show not only that gaps can be observed, but also that they are replicable. Instead, our interest lies in the interpretation of observed gaps. The primary conclusion derived from the data reported here is that observed WTP-WTA gaps do not reflect a fundamental feature of human preferences. That is, endowment effect theory does not seem to explain observed gaps. In addition, our results suggest that observed gaps should not be interpreted as support for prospect theory.

A review of the literature reveals that WTP-WTA gaps are not reliably observed across experimental designs. Given the nature of reported experimental designs, we posited that differences in experimental procedures might account for the differences across reported results. This conjecture prompted us to develop procedures to test for the robustness of the phenomenon. We conducted comparative experiments using procedures commonly used in studies that report observed gaps (i.e., KKT). We also employed a "revealed theory" methodology to identify procedures reported in the literature that provide clues about experimenter notions regarding subject misconceptions. We then conducted experiments that implemented the union of procedures used by experimentalists to control for subject misconceptions. The comparative experiments demonstrate that WTP-WTA gaps are indeed sensitive to experimental procedures. By implementing different procedures, the phenomenon can be turned on and off. When procedures used in studies that report the gap are employed, the gap is readily observed. When a full set of controls is implemented, the gap is not observed.

The fact that the gap can be turned on and off demonstrates that interpreting gaps as support for endowment effect theory is problematic. The mere observation of the phenomenon does not support loss aversion-a very special form of preferences in which gains are valued less than losses. That the phenomenon can be turned on and off while holding the good constant supports a strong rejection of the claim that WTP-WTA gaps support a particular theory of preferences posited by prospect theory. Loss aversion might in some sense characterize preferences, but such a theory most likely does not explain observed WTP-WTA gaps. Exactly what accounts for observed WTP-WTA gaps? The thesis of this paper is that observed gaps are symptomatic of subjects' misconceptions about the nature of the experimental task. The differences reported in the literature reflect differences in experimental controls for misconceptions as opposed to differences in the nature of the commodity (e.g., candy, money, mugs, lotteries, etc.) under study.

The researchers aren't entirely sure why speaking in a less familiar tongue makes people more "rational", in the sense of not being affected by framing effects or loss aversion. But they think it may have to do with creating psychological distance, encouraging systematic rather than automatic thinking, and with reducing the emotional impact of decisions. This would certainly fit with past research that's shown the emotional impact of swear words, expressions of love and adverts is diminished when they're presented in a less familiar language.

Paywalled article (can someone with access throw a PDF up on dropbox or something?): http://pss.sagepub.com/content/early/2012/04/18/0956797611432178

Blog summary: http://bps-research-digest.blogspot.co.uk/2012/06/we-think-more-rationally-in-foreign.html

Mr. Malfoy is new to the business of having ideas, and so when he has one, he becomes proud of himself for having it. He has not yet had enough ideas to unflinchingly discard those that are beautiful in some aspects and impractical in others; he has not yet acquired confidence in his own ability to think of better ideas as he requires them. What we are seeing here is not Mr. Malfoy's best idea, I fear, but rather his only idea.

- Harry Potter and the Methods of Rationality

I want to emphasize yet again that the tools [described in Serious Creativity] are deliberate and can be used systematically. It is not a matter of inspiration or feeling in the mood of being "high." You can use the tools just as deliberately as you can add up a column of numbers.

- Edward De Bono, Serious Creativity

I will summarize some of the techniques for how to generate ideas presented in Serious Creativity. The book also has other material, e.g. interesting deep theories about why these techniques work, arguments for the importance of creativity, and more techniques beyond what's described in this post, but in the interest of keeping this post concise and useful, I will only describe one kind of technique and urge you to just try it. You should read the book if you want more detail or techniques.

These techniques can be used both when you have a problem you need to solve and when you have a general area that you suspect could be improved or innovated, but don't have any specific ideas of what's wrong (or even if you don't feel like there's anything wrong at all).

The technique I will describe in this post is that of "provocation" followed by "movement." A provocation is a seemingly random or nonsensical sentence or phrase. Movement is the process of going forward with a provocation and actually generating an idea. There are precise, formal techniques for generating provocations and movement, which I will describe after giving an example of how this "provocation-movement" process works.

Example

Provocation: Planes land upside down.
Movement: We can imagine this actually happening, and observe that the pilot would have a better view of the landing area. This naturally leads us to consider other ways to improve the pilot's view of the landing area. Perhaps we could move the cockpit to the bottom, or add video cameras. So using this technique, we've identified an area for improvement and two possible ways to make that improvement.

Setting Up Provocations

Provocation is a way to avoid getting stuck in the same "mental pathways" (see priming) so that you can find new ones. Provocations should not make sense and are not necessarily intended to convey meaning; they are just intended to "make things happen in our minds." The book precedes provocations with "po," a word used to indicate that the sentence is intended to be nonsensical and illogical. Po stands for "provoking operation." The book describes several techniques for generating provocations.

1. Escape method: Think of something that we take for granted, and negate it. E.g., "Po, restaurants do not have food" or "Po, shoes do not have soles."
2. Reversal: Take a standard arrangement or relationship that we take for granted, and reverse it. E.g. "I have orange juice for breakfast" becomes "Po, the orange juice has me for breakfast". Note that the reversal would not be "Po, I do not have orange juice for breakfast." That would be the escape method.
3. Exaggeration: Suggest that some dimension or measurement falls far outside its normal range (either greater or lesser). E.g. "Po, every household has 100 phones" or "Po, the phone has 1 dialing button." If you're making the dimension smaller, do not bring it to 0 or you're just using the escape method again. E.g. "Po, the phone has 0 dialing buttons" is not an exaggeration, it's an escape.
4. Distortion: Take normal arrangements (e.g. relationships or time sequences) and switch them around. E.g. "Po, you close the letter after you post it," "Po, criminals pay for the police force," or "Po, food prepares customers for chefs."
5. Wishful thinking: "Wouldn't it be nice if..." put forward a fantasy that is known to be impossible. E.g. "Po, the pencil should write by itself."

A provocation doesn't need to follow from one of these techniques. A provocation can be any incorrect or absurd statement. These techniques are just easy step-by-step ways to generate provocations without requiring any elusive "spark of inspiration." Once a provocation is generated, it should be followed by one or more of the movement techniques described in the next section.

If you are trying to solve a specific problem or innovate in a particular domain, then choose provocations related to the domain. That is, if you're trying to figure out how to improve wikipedia, don't use a provocation like "Po, the orange juice has me for breakfast," choose one like "Po, citations are not needed" (escape) or "Po, articles contain encyclopedias." (reversal).

Movement

Movement allows you to take some idea, concept, or provocation and move forward with it to generate more useful ideas and concepts. These techniques don't apply solely to provocations: you can use them for ideas and concepts too. The book describes 5 formal techniques for movement:

1. Extract a principle: Focus on some principle of the provocation, and then work with that principle to discover other ideas related to it. E.g. with the provocation "Po, bring back the town crier", we may extract the principle that the town crier can go to where people are, and then we try to generate ideas related to that principle.
2. Focus on the difference: Compare the provocation to existing ways of doing things. How are they different? Then you can consider other ways to use this difference. This is very similar to "extract a principle."
3. Moment to moment: imagine what would happen if the provocation were put into effect. We are not interested in the final effect, but the moment-to-moment happenings. E.g. for "Po, orange juice has me for breakfast", you may imagine yourself falling into a giant glass of orange juice.
4. Positive aspects: Look directly for benefits. What are the positive aspects of the provocation? Once you've identified some positive aspects you can consider if you can achieve some of them in other ways (again, this is similar to extract a principle, it's just another way of thinking about it).
5. Circumstances: In what circumstances would the provocation have immediate value? E.g. for the provocation "Po, drinking glasses should have rounded bottoms," you could notice that this would be useful if you didn't want people to be able to put down their glasses. This could be good for bars, where you want people to drink more and faster.

You can use these movement techniques not just on provocations, but also ideas or concepts. For example, you may start with a provocation, use the "moment to moment" technique which gives you an idea, and then you could use the "positive aspects" technique with that idea to generate more ideas. Also, of course, you do not need to strictly use just these techniques. If a provocation directly leads you to think of something interesting without explicitly choosing to use one of these techniques, that's fine, you should explore the idea more. Use these when you need them.

More Examples

Here's another example from the book. This one uses the "moment to moment" movement technique:

Po, cars have square wheels

We imagine a car with square wheels. We imagine this car starting to roll. The square wheel rises up on its corner. This would lead to a very bumpy ride. But the suspension could anticipate this rise and could adjust by getting shorter. This leads to the concept of an adjusting suspension. This in turn leads to the idea of a vehicle for going over rough ground. A jockey wheel would signal back the state of the ground to the suspension which would then adjust so that the wheel was raised to follow the "profile" of the ground...This was an idea I first suggested about twenty years ago. Today several companies such as Lotus (part of GM) are working on "intelligent suspension" which behaves in a similar way.

And here's another one from the book. The provocation uses the "escape" method and the movement seems to use the "circumstances" method:

Po, waiters are not polite.

This leads to an idea for waiters to be actors and actresses. The menu indicates the "character" of the waiter. You can order whichever waiter you wanted: belligerent, humorous, obsequious, and so on. You might order a belligerent waiter and enjoy having a fight with him. The waiters and waitresses would act out the assigned role.

Warnings

• As a general principle, try to avoid saying "oh, but this is just like this other existing product" whenever you generate an idea. Usually it's not just like the existing idea, you're just interpreting it in that way because we naturally follow paths toward the familiar. So if you have a half-formed idea that could take several directions, fight the urge to immediately take it down an existing path and then discard it because it already exists. Leave it in the half-formed stage instead. I'm reminded of the concept of semantic stopsigns. Saying an idea is "the same as" something else gives the illusion of having fully explored the idea, when in reality you just jumped immediately to one possible development (possibly the least useful development, since it's one you know already exists).
• Similarly, do not take too many steps when moving from a provocation. This will just lead you to an existing idea. There's nothing to be gained by playing 6 degrees of separation with provocations and existing ideas. Just take a few small steps. If nothing comes to you, try other movement techniques or try a different provocation. 
• You're not expected to come up with a good idea for every provocation. Most of the time you'll come up with some mediocre or half-formed idea, or even no idea at all. This is fine.
• You should write down anything you come up with that seems interesting (even if it's a bad idea in its current form, if it has something interesting about it, write it down) and then come back to it later and think about it more (either using these techniques or just your normal thinking processes for improving and adapting ideas).

Fermi problem

In science, particularly in physics or engineering education, a Fermi problem, Fermi question, or Fermi estimate is an estimation problem designed to teach dimensional analysis, approximation, and the importance of clearly identifying one's assumptions. Named after physicist Enrico Fermi, such problems typically involve making justified guesses about quantities that seem impossible to compute given limited available information.

Fermi was known for his ability to make good approximate calculations with little or no actual data, hence the name. One example is his estimate of the strength of the atomic bomb detonated at the Trinity test, based on the distance travelled by pieces of paper dropped from his hand during the blast. Fermi's estimate of 10 kilotons of TNT was remarkably close to the now-accepted value of around 20 kilotons, a difference of less than one order of magnitude.

[...]

Scientists often look for Fermi estimates of the answer to a problem before turning to more sophisticated methods to calculate a precise answer. This provides a useful check on the results: where the complexity of a precise calculation might obscure a large error, the simplicity of Fermi calculations makes them far less susceptible to such mistakes. (Performing the Fermi calculation first is preferable because the intermediate estimates might otherwise be biased by knowledge of the calculated answer.)

Fermi estimates are also useful in approaching problems where the optimal choice of calculation method depends on the expected size of the answer. For instance, a Fermi estimate might indicate whether the internal stresses of a structure are low enough that it can be accurately described by linear elasticity; or if the estimate already bears significant relationship in scale relative to some other value, for example, if a structure will be over-engineered to withstand loads several times greater than the estimate.

Although Fermi calculations are often not accurate, as there may be many problems with their assumptions, this sort of analysis does tell us what to look for to get a better answer.

Link: en.wikipedia.org/wiki/Fermi_problem

Fermi Problem: Power developed at the eruption of the Puyehue-Cordón Caulle volcanic system in June 2011

Enrico Fermi was renowned for his ability to make reliable estimates. But how well can you do on a modern estimation problem?

[...]

Hernan Asory and Arturo Lopez Davalos at the Comision Nacional De Energia Atomica in Argentina, have set themselves (and their students) a similar estimation task. The problem is to estimate the energy release as well as the volume and mass of sand ejected during the eruption of the Puyehue-Cordon Caulle volcano in Chile on 4 July.

You can look up the calculations and the assumption they make in the paper. You might want to try the estimate yourself.

Link: technologyreview.com/blog/arxiv/27140/

If you want to get better at doing rough mental calulcations, the following books might provide some valuable heuristics:

Street-Fighting Mathematics: The Art of Educated Guessing and Opportunistic Problem Solving

Time for some quick arithmetic: Is 3600 x 4.4 x 10⁴ x 32 larger or smaller than 3 x 10⁹?

Finding the right answer, says Sanjoy Mahajan, associate director for teaching initiatives at MIT’s Teaching and Learning Laboratory, does not require crafting a long, tedious calculation. Instead, the key to solving this problem — and many others — lies in having informal tools on hand that let us attack the problem. Though the result may not be perfectly precise, he believes, intuitive mathematical reasoning is often sufficient for our needs.

“That’s not to say exact answers aren’t useful,” says Mahajan, “but if looking for them is your only approach, you may never get any answer at all. Sometimes it’s better to start with something rough.”

[...]

Mahajan believes we should learn practical math tools and understand why they work.

[...]

Mahajan’s unconventional teaching practices stem from his focus, as a physicist, on finding quick, practical answers. Then again, perhaps rolling up one’s sleeves and hacking through problems is how everyone works. “There is a culture in pure mathematics that emphasizes rigor and careful proofs,” says Strogatz. “Yet all practicing mathematicians know we also use our intuitions, then we clean our answers up.”

[...]

So let’s get back to the initial question (the numbers relate to the storage capacity of a data CD-ROM). The key to solving it, says Mahajan, is to recognize that the components of the first, messy-looking number can be broken into powers of 10. Then we can temporarily set aside these powers of 10 — Mahajan calls this “taking out the big part,” one of his tenets of problem-solving — while handling the smaller, simpler multiplication problem.

Okay: Picture the number as (3.6 x 10³) x (4.4 x 10⁴) x (3.2 x 10¹). To multiply powers of 10 in practice, we add them, here producing 10⁸. Leave that aside momentarily and multiply 3.6 x 4.4 x 3.2. The answer is about 50, or 5.0 x 10¹. Combine that with 10⁸, and we have our answer: Roughly 5.0 x 10⁹, which is bigger than 3 x 10⁹. Street-fighting math, and we barely got a scratch. 

Link: web.mit.edu/newsoffice/2010/street-fight-0329.html

Secrets of Mental Math: The Mathemagician's Guide to Lightning Calculation and Amazing Math Tricks

Yes, even you can learn to do seemingly complex equations in your head; all you need to learn are a few tricks. You’ll be able to quickly multiply and divide triple digits, compute with fractions, and determine squares, cubes, and roots without blinking an eye. No matter what your age or current math ability, Secrets of Mental Math will allow you to perform fantastic feats of the mind effortlessly. This is the math they never taught you in school.

For humans problems can seem intractable for a long time and then suddenly become easy. Forming a coherent chemistry was taking a very long time until Lavoisier thought to look at the mass of reactants and products. And then we had a periodic table 100 years after that. Compare this to little progress from having bounced around in alchemy for 2000 years or so.

So identifying and understanding angles of attacks is important for tackling the thorny problems that face us today.