What plans could a prospective cryonicist try out, beyond simply signing up, that could increase the odds of eventually having a pleasant re-animation experience?

To show what I mean, here are the main ideas I've managed to come up with so far. None of these particular ideas are a standard part of a cryonics preservation package. Some are easier to implement than others, some are more likely to have an effect than others, some have potentially greater effect than others.

* Arranging for as much information about oneself (photo albums, emails, grade school report cards, etc) as possible to be placed on archival media and stored along with one's body. Reasoning: If the cryo-preservation procedure causes brain damage, and technology advances sufficiently before re-animation, then this information potentially allows for that damage to be at least partially reconstructed.

* Requesting additional data about the cryo-preservation procedure used on oneself be archived. Eg, requesting that, to whatever degree doesn't interfere with the procedure, it be videoed.

* Making arrangements for an animal body to be cryo-preserved with the same procedure one's own body was preserved with. A lab chimp would be ideal, but difficult to arrange for a number of reasons; more likely, a more common animal of around human mass would be feasible, such as a dog or goat. Even a few lab-rats might help. Reasoning: It gives future re-animators an additional opportunity to experiment with re-animation techniques, before attempting to re-animate a person.

* Noting down one's preferences and requests for future re-animators. Eg, from "I'd appreciate having a cat nearby to pet and calm down as I wake up" to "If you have to rebuild my body from scratch anyway, and it's within cultural norms, I would appreciate being gender _____" to "If you create a digital/electronic/computer/data copy of my mind, I would like a copy of that to be placed in offline, air-gapped storage, so that if every active copy of my mind is destroyed, there will always be that original backup available to re-instantiate myself." Or just more general ideas, such as, "My goal is to live forever, and I would prefer whatever means most likely lead to that happening to be tried."


I'm not nearly as creative as I wish I could be; so I'm hoping that the local group-mind here might be able to offer further ideas, or improvements or refinements to the above ones.

So: What extras can you think of?

Mr. Malfoy is new to the business of having ideas, and so when he has one, he becomes proud of himself for having it. He has not yet had enough ideas to unflinchingly discard those that are beautiful in some aspects and impractical in others; he has not yet acquired confidence in his own ability to think of better ideas as he requires them. What we are seeing here is not Mr. Malfoy's best idea, I fear, but rather his only idea.

- Harry Potter and the Methods of Rationality

I want to emphasize yet again that the tools [described in Serious Creativity] are deliberate and can be used systematically. It is not a matter of inspiration or feeling in the mood of being "high." You can use the tools just as deliberately as you can add up a column of numbers.

- Edward De Bono, Serious Creativity

I will summarize some of the techniques for how to generate ideas presented in Serious Creativity. The book also has other material, e.g. interesting deep theories about why these techniques work, arguments for the importance of creativity, and more techniques beyond what's described in this post, but in the interest of keeping this post concise and useful, I will only describe one kind of technique and urge you to just try it. You should read the book if you want more detail or techniques.

These techniques can be used both when you have a problem you need to solve and when you have a general area that you suspect could be improved or innovated, but don't have any specific ideas of what's wrong (or even if you don't feel like there's anything wrong at all).

The technique I will describe in this post is that of "provocation" followed by "movement." A provocation is a seemingly random or nonsensical sentence or phrase. Movement is the process of going forward with a provocation and actually generating an idea. There are precise, formal techniques for generating provocations and movement, which I will describe after giving an example of how this "provocation-movement" process works.

Example

Provocation: Planes land upside down.
Movement: We can imagine this actually happening, and observe that the pilot would have a better view of the landing area. This naturally leads us to consider other ways to improve the pilot's view of the landing area. Perhaps we could move the cockpit to the bottom, or add video cameras. So using this technique, we've identified an area for improvement and two possible ways to make that improvement.

Setting Up Provocations

Provocation is a way to avoid getting stuck in the same "mental pathways" (see priming) so that you can find new ones. Provocations should not make sense and are not necessarily intended to convey meaning; they are just intended to "make things happen in our minds." The book precedes provocations with "po," a word used to indicate that the sentence is intended to be nonsensical and illogical. Po stands for "provoking operation." The book describes several techniques for generating provocations.

1. Escape method: Think of something that we take for granted, and negate it. E.g., "Po, restaurants do not have food" or "Po, shoes do not have soles."
2. Reversal: Take a standard arrangement or relationship that we take for granted, and reverse it. E.g. "I have orange juice for breakfast" becomes "Po, the orange juice has me for breakfast". Note that the reversal would not be "Po, I do not have orange juice for breakfast." That would be the escape method.
3. Exaggeration: Suggest that some dimension or measurement falls far outside its normal range (either greater or lesser). E.g. "Po, every household has 100 phones" or "Po, the phone has 1 dialing button." If you're making the dimension smaller, do not bring it to 0 or you're just using the escape method again. E.g. "Po, the phone has 0 dialing buttons" is not an exaggeration, it's an escape.
4. Distortion: Take normal arrangements (e.g. relationships or time sequences) and switch them around. E.g. "Po, you close the letter after you post it," "Po, criminals pay for the police force," or "Po, food prepares customers for chefs."
5. Wishful thinking: "Wouldn't it be nice if..." put forward a fantasy that is known to be impossible. E.g. "Po, the pencil should write by itself."

A provocation doesn't need to follow from one of these techniques. A provocation can be any incorrect or absurd statement. These techniques are just easy step-by-step ways to generate provocations without requiring any elusive "spark of inspiration." Once a provocation is generated, it should be followed by one or more of the movement techniques described in the next section.

If you are trying to solve a specific problem or innovate in a particular domain, then choose provocations related to the domain. That is, if you're trying to figure out how to improve wikipedia, don't use a provocation like "Po, the orange juice has me for breakfast," choose one like "Po, citations are not needed" (escape) or "Po, articles contain encyclopedias." (reversal).

Movement

Movement allows you to take some idea, concept, or provocation and move forward with it to generate more useful ideas and concepts. These techniques don't apply solely to provocations: you can use them for ideas and concepts too. The book describes 5 formal techniques for movement:

1. Extract a principle: Focus on some principle of the provocation, and then work with that principle to discover other ideas related to it. E.g. with the provocation "Po, bring back the town crier", we may extract the principle that the town crier can go to where people are, and then we try to generate ideas related to that principle.
2. Focus on the difference: Compare the provocation to existing ways of doing things. How are they different? Then you can consider other ways to use this difference. This is very similar to "extract a principle."
3. Moment to moment: imagine what would happen if the provocation were put into effect. We are not interested in the final effect, but the moment-to-moment happenings. E.g. for "Po, orange juice has me for breakfast", you may imagine yourself falling into a giant glass of orange juice.
4. Positive aspects: Look directly for benefits. What are the positive aspects of the provocation? Once you've identified some positive aspects you can consider if you can achieve some of them in other ways (again, this is similar to extract a principle, it's just another way of thinking about it).
5. Circumstances: In what circumstances would the provocation have immediate value? E.g. for the provocation "Po, drinking glasses should have rounded bottoms," you could notice that this would be useful if you didn't want people to be able to put down their glasses. This could be good for bars, where you want people to drink more and faster.

You can use these movement techniques not just on provocations, but also ideas or concepts. For example, you may start with a provocation, use the "moment to moment" technique which gives you an idea, and then you could use the "positive aspects" technique with that idea to generate more ideas. Also, of course, you do not need to strictly use just these techniques. If a provocation directly leads you to think of something interesting without explicitly choosing to use one of these techniques, that's fine, you should explore the idea more. Use these when you need them.

More Examples

Here's another example from the book. This one uses the "moment to moment" movement technique:

Po, cars have square wheels

We imagine a car with square wheels. We imagine this car starting to roll. The square wheel rises up on its corner. This would lead to a very bumpy ride. But the suspension could anticipate this rise and could adjust by getting shorter. This leads to the concept of an adjusting suspension. This in turn leads to the idea of a vehicle for going over rough ground. A jockey wheel would signal back the state of the ground to the suspension which would then adjust so that the wheel was raised to follow the "profile" of the ground...This was an idea I first suggested about twenty years ago. Today several companies such as Lotus (part of GM) are working on "intelligent suspension" which behaves in a similar way.

And here's another one from the book. The provocation uses the "escape" method and the movement seems to use the "circumstances" method:

Po, waiters are not polite.

This leads to an idea for waiters to be actors and actresses. The menu indicates the "character" of the waiter. You can order whichever waiter you wanted: belligerent, humorous, obsequious, and so on. You might order a belligerent waiter and enjoy having a fight with him. The waiters and waitresses would act out the assigned role.

Warnings

• As a general principle, try to avoid saying "oh, but this is just like this other existing product" whenever you generate an idea. Usually it's not just like the existing idea, you're just interpreting it in that way because we naturally follow paths toward the familiar. So if you have a half-formed idea that could take several directions, fight the urge to immediately take it down an existing path and then discard it because it already exists. Leave it in the half-formed stage instead. I'm reminded of the concept of semantic stopsigns. Saying an idea is "the same as" something else gives the illusion of having fully explored the idea, when in reality you just jumped immediately to one possible development (possibly the least useful development, since it's one you know already exists).
• Similarly, do not take too many steps when moving from a provocation. This will just lead you to an existing idea. There's nothing to be gained by playing 6 degrees of separation with provocations and existing ideas. Just take a few small steps. If nothing comes to you, try other movement techniques or try a different provocation. 
• You're not expected to come up with a good idea for every provocation. Most of the time you'll come up with some mediocre or half-formed idea, or even no idea at all. This is fine.
• You should write down anything you come up with that seems interesting (even if it's a bad idea in its current form, if it has something interesting about it, write it down) and then come back to it later and think about it more (either using these techniques or just your normal thinking processes for improving and adapting ideas).

Note: The following outline of my research proposal is unfinished. I posted it in the discussion section to spur conversation and get constructive criticism (successfully, I might add). If you have any suggestions, then please make them. I will be monitoring the discussion and improving the proposal until I feel it is ready to be posted as a main article.

Introduction

I think I may have found a novel use for an old technique, which may or may not have implications for rational decision making. I am open to constructive criticism or even deconstructive criticism if you make a sound argument. Ultimately, I would like the experiment to be put to the test. If you have the supplies and know-how to carry it out, then feel free to try it and report your findings.

The Goal:

•  Catalyze the brainstorming process in a way that increases both the number and quality of ideas made.

Methods:

• Find a problem that needs solving. Unifying general relativity and quantum mechanics is a good, but ambitious, example. Some more likely problems that could be solved are: "How might I solve my relationship problems," or "how can I advertise my company's product to its target demographic", or "what are some ideas to make quick money."
• Find 2-3 rationalists who understand the problem well. They don't need to be expert rationalists; the most important part is that they know the difference between rationalization and rationality. In the QM example above, and in most scientific applications of the method, all players should have access to the experimental data.
• Assign two of the three rationalists to the "brainstormers" group (name subject to change), whose primary concern is to make logical connections between the data to form hypotheses.
• Assign the odd-rationalist-out as the Confessor, whose primary concern, like in TWC, is to preserve sanity. It is the Confessor's job to catch the brainstormers when they make a logical leap or use biased reasoning. Some tactics the Confessor might use are the rationalist taboo, the reversal test, and argument from the least convenient world.
• This is a scaled up version of what the brain seems to do. We need the brainstormers and the Confessor to act the part of the Apologist and the Revolutionary, respectively.
• The Confessor - brainstormer dynamic is interesting in its own right, but I believe it can be improved. Now bear with me, because the optional step is for the brainstormers to smoke Cannabis. Not too much, but just enough so that connections between ideas are more quickly apparent to them. Remember, the goal is to have the brainstormers make many connections. They need to output quantity over quality, while the Confessor picks out anything that is quality and gently guides the brainstormers toward more quality ideas. Think of it like r-selected evolution.

• Ideally, we would split twelve rationalists into four groups of three (the alternative is to use the same group repeatedly). Group 1 would be told to just brainstorm the problem. Group 2 would be told to choose one among them to be the Confessor. Group 3 and 4 would be told the same, but their brainstormers would be given either Cannabis or a placebo.
• A placebo can be made by extracting the cannabinoids using ethanol or glycerine. All that should be left after extraction is plant matter, and the tincture can be used later for medicinal or recreational purposes. The placebo Cannabis and active Cannabis will have to be rolled into joints because extraction removes some of the plant's pigmentation. If you have access to a lab, then you might follow  this procedure  for the extraction; otherwise, use  this guide  for doing the extraction at home.
• If you don't want to go to the trouble of making the placebo, then you may skip the control group and only do groups 1 - 3. It would be nice to get some preliminary data, even if skewed slightly by the placebo effect.
• For data collection, the Confessor will note down any idea made by the brainstormers, marking the ones which were discarded. After a given amount of time, enumerate the data and compare the groups. The hypothesized result is that the smoking group will make the greatest quantity of ideas, followed by the non-smoking partitioned group, followed by the normal brainstorming group. It is also hypothesized that the smoking group will make the greatest quality ideas, due to a combination of the highly creative nature of ideas made while high (explained below) and the Confessor's job of immediately discrediting any faulty reasoning.

Some evidence that the Cannabis route might be a good one to pursue (more references to be added):

• There is evidence that Cannabis engages the mind in semantic Hyper-Priming^1,2, meaning that distantly-related concepts are primed quickly after having been exposed to an idea. For instance, a smoker might quickly respond to the word "fish" with "submarine," whereas someone who is sober might respond with "fin." If I understand it correctly, then this doesn't mean the smoker cannot answer "fin," only that the more distantly related concepts are given a higher priority than they normally would. One can see why this might be advantageous for brainstorming, but I suggest to take my - and mindhack.com's - interpretation of the paper with a grain of salt until someone with access can read it in full.
• *Cannabis allows erroneous perspectives to be rapidly dismissed in the light of new evidence. While high, it is easy to put one's pride aside and say "oops". This is made especially easy if the user has cached understanding of the art of rationality. In other words: they will listen to the Confessor.* <-- (I haven't found any literature to support this claim, yet. It seems true in my experience, but it might not be true for everyone. If the brainstormers prove to be too clingy, we could alter the method by changing the Confessor's name to Kiritsugu and having the brainstormers agree to always defer to the Kiritsugu's better judgement. The Kiritsugu will have to take care to examine its own judgement and only discard the truly irrational ideas).

Anecdotal evidence:

• Artists, writers, and even scientists have long used Cannabis and other psychoactive drugs as a tool to make "insights." I'm defining insight as the connection and/or creation of ideas (erroneous or otherwise), possibly due to hyper-priming. The Confessor, in the early pioneers' case, was usually their sober self. As Hemingway wrote, "write drunk; edit sober."
• Less gifted stoners have been doing this for ages  but they - for the most part - are completely undisciplined, believe in dubious pseudoscience, and/or don't have a rational observer to moderate them.
• This is going a bit meta, but the outline to the outline of this idea was made while I was high. It was the first time I smoked since having been introduced to Less Wrong and "The Way", and I was surprised to find that I still had most of my wits about me. Although I would often begin down paths that were just Rationalizations, I usually caught myself. In the instances where I didn't catch myself, and it seemed like a legitimately good insight, I wrote the idea down for future (sober) consideration.
• One of the good, practical, non-meta insights I made that night was a life plan. My plan up until this point had been to finish my undergraduate degree and then immediately go to grad school, relying on my schooling and a bit of luck to maybe hopefully turn into a somewhat-successful scientist somewhere along the road. The problem is that I suffer from quite a bit of procrastination, in part because I don't know exactly what I want to do. I don't have any strong passions or any real motivation. My college career, so far, has been an uphill battle against crippling akrasia.
• Aided by Cannabis, I finally saw the obvious: I need to make an effort to find a passion. My new plan is to get a job as a computer programmer after finishing undergrad, but to continue self-teaching in Biology and other sciences. I've already taken the first step by having Computer Science as my minor, and I can help my resume along right now by getting involved in open source projects. As for self-teaching, that's made easy by open courseware like that found on Khan Academy, MIT, and other places, and I always have the old-fashioned solution of just reading textbooks. After following my interests for a while and learning what things I really, really like to learn about, then I'll go to grad school with an actual PhD thesis in mind and money in the bank.
• I'm attributing these insights (the life plan, some other ideas I'm not mentioning, and even the hypothesis itself) to hyper-priming and later editing, but they might have just been made because I was focused on the problems. Hence the need for an experimentally-controlled test.

Conclusion

• Cannabis allows connections to be made between concepts which normally seem unrelated. This is an experience commonly reported by users, and experimentally verified. Some of these connections will inevitably be false, but others might be true, and a third party - a Confessor - might be able to distinguish truth from falsehood. Whether the Confessor - brainstormer dynamic is any more efficient or productive than a normal brainstorming session is an open question, and the only way to really know is to test the hypothesis.

References

What I'm missing. To be included later:

• References to the benefits and techniques of traditional brainstorming. In lieu of that, for now, here's  this  and  this .
• More references to Cannabis research.

Introduction

It is common knowledge that many artists have used drugs (alcohol, opiates, cannabis, LSD, ...) and that this account for part of their creativity. This common knowledge is usually opposed to people advocating rationality in sentences like "but with only your rationality, we wouldn't have much art", "you need chaos to make art" or even "the best artists were that great because they were irrational".

Eliezer partly addressed the issue in the lawful intelligence Sequence. While this Sequence is very interesting, I feel it didn't completely address the issue (unlike most of the Sequences). My hypothesis is that it's mostly focused on what's important to building a Friendly AI (which is a worthy goal, this should not be taken as a criticism), not so much as explaining creativity in actual humans. So I'm writing an article with my current thoughts on the topic, and I would welcome any additional argument, hypothesis, research paper, ... that anyone from the LW community can point me to. This article is not supposed to come to any definitive conclusion, but to show my current state of thinking on that issue. I hope to both give and receive in writing it.

Reasons for which it could be an illusion

Availability bias

The first question to ask about "it is a common knowledge that many artists were using drugs" is : is this common knowledge true, or not, and if not, why do so many people believe something which is false ?

Availability bias comes will full power on this issue : when we hear that a given artist (musician, writer, painter, ...) was taking drugs, we add a "drug addict" tag to him. Or more accurately we create a link between the "drug addict" node and his node in our belief network. When asked about artists who did take drugs, we can easily state many names : for example Hemingway, Van Gogh, the Beatles. When asked about artists who didn't take drugs... well, we usually don't have "did not take drug" node in our belief networks, and no easy way to say that Asimov or Bach didn't take drugs.

Even when doing specific research, we can know with almost absolute certainty that Hemingway was drinking a lot of alcohol, but not so confidently that Asimov didn't. It's easier to be sure of the existence of something, than to be sure of its non-existence.

Reverse causality

The second question, if even after considering the affect of availability bias, it still seems than artists take drugs more often than average, is to ask about which sens the causality flows. Statistical correlation points to a causality, but doesn't tell you which sense is the causality, nor if it's direct or indirect.

There can be many reasons for which the causality works backwards : someone is not a good artist because he takes drug, but he takes drugs because he is an artist.

The lifestyle of a professional artist is usually different from the lifestyle of most other people. They usually don't have to wake up at 7 to be at work at 8, since they can work at any time. They also tend to be either very poor (many artists were only praised and recognized after being dead) or very rich (for the few who reach success while they are still alive). And we know that very poor people tend to fall on alcohol more often, while very rich people tend to use more frequently some of the very expensive drugs like cocaine.

Being an artist also usually induces a higher uncertainty about the future than with most regular jobs, which may trigger the use of drugs to make the angst easier to withstand.

Common cause

Apart from direct causality one way or another, a statistical correlation can also indicate that is a hidden common cause between the two phenomenas. If artists take drugs more often, it could be because there is a common reason that pushes people to both by a great artist and to take drugs.

Many reasons can be invoked that way, due to superexponential hypothesis space. I'll risk to be privileging the hypothesis but I can name a few. For example someone with an overdevelop emotional sensitivity could be both great at writing art able to call to our emotions and be more tempted to use drugs as relief from over-experienced negative emotions. Or someone who happens to be an outcast can be more likely to perform art (since it is usually a solitary work, not a team work) and at the same time use more drugs to escape from the pain of being an outcast.

So, where do we stand now ?

When faced with a statement such as "artists take drugs more often than average, so drugs help creativity" we can emit 4 different classes of hypothesis :

1. The initial statement is wrong, artists don't take more drugs than average.
2. Artists take more drugs than average, but the causality is reverse (it's being an artist that make you take drugs, not the other way around).
3. Artists take more drugs than average, but that's because of a common factor that increases likelihood of taking drugs and of making great art, not the drugs themselves increasing artistic creativity.
4. This is true, for a reason or another, drugs help creativity.

We saw some possible reasons for 1., 2. and 3. Some of them seem to be very real to me, especially the availability bias, but I do not think they totally account for the facts.

As much as I would love to be able to stop here and say that drugs and chaos play no positive role in creativity, that creativity is purely lawful and rational, I fear that would be wishful thinking and refusing to attack my belief's weak points. To state it more lightly : my D&D alignment could very well be lawful-good (as my friends tease me it is), but that shouldn't prevent me from admitting that chaos play a positive role somewhere if it actually does.

Reasons for which it could be real

Chaos and optimization

Generating great art can be seen as an optimization process. The actual function that evaluate a piece of art may be very complex, partly depending of the recipient, and its formalization unknown, but it can still be considered an optimization process : generating a book, or a painting, or a song that scores very high in most people's evaluation function.

In general, chaos is not an optimization process. Adding chaos to an optimization process usually makes it worse. But there are known counter-examples, where an imperfect optimization process will gain from a slight controlled increase of chaos.

Lawfully controlled chaotic optimization

The first known example is the first optimization process ever : evolution. Evolution involves too part : mutations which are chaotic, done at random, and natural selection which is lawful and selects the few evolutions that happened to be positive. The Roger Zelazny picture of the universe being an equilibrium between Order and Chaos may come from that pattern. If you increase chaos too much in natural selection, the information will not be replicated enough from generation to generation, and not much optimization will occur. But if you don't have any mutation, if you remove all the chaos, the process will freeze too.

I remember an experiment from biology lessons in high school : take two small boxes of glass, put cotton with water and sugar at the bottom. Take some bacteria, and but the bacteria on side of the box. Take an antibiotics pill, and put it on the other side of the box. Put box A in safe storage. Put box B in safe storage too, but every day, expose it to a small amount of UV light. The bacteria of box A will quickly spread on the cotton, but will not go anywhere close the antibiotics pill. The bacteria of box B will start doing the same, but after two or three weeks, they conquer even the antibiotics area. After a longer time period, box A bacteria will also overcome the antibiotics, but it'll take them much longer. The UV light increased the mutation rate, and sped up the optimization process of evolution. But only a very small dose of UV light does that, overdose it, and the bacteria B will all die.

That's what I call "lawfully controlled chaotic optimization" : there is a lawful control process (here, natural selection) that selects randomly tried solutions. That's something that can directly apply to artists : the control process (be it the filter of editors/publishers, or the filter of public reaction) is, to a point, lawful, but the process that generate solutions could benefit from a slight increase in chaos. Or more exactly, the combined (generator + filter) algorithm could perform better with a slightly more chaotic generator. To retake Eliezer's definition of creativity, which was "the creative surprise is the idea that ranks high in your preference ordering but low in your search ordering", adding chaos to preference ordering would be pointless, but adding chaos to the search ordering can allow more creative surprise to happen in a given finite time.

There is still a major difference between the two processes described here (evolution and human creativity) : evolution uses a fully random generator, whereas the human brain has a great ability of generating non-random designs, allowing a much faster improvement rate. You'll never get a book of Hemingway or a painting of Van Gogh by randomly selecting letters or randomly putting paint on a canvas. The chance of that is too infinitesimally low. So the generator will have to stay mostly lawful. Hemingway used words and respected the rules of grammar. Van Gogh painted something which look very like real sunflowers. A fully chaotic process would never produce anything near their masterpieces even given billions of years. So artistic creativity must be mostly lawful, even for generating its hypothesis to select from.

As spotted by Vaniver in the comments, Hemingway himself said something very similar to that thesis : "Write drunk; edit sober."

Avoiding local minimal

One big problem of optimization processes is local minimal. Most of the naive optimization process, like a gradient descent, will get trapped into local minima. Let's have a look at that curve (burrowed from Wikipedia) :

If you start a naive optimization algorithm in the right part of the curve, you'll very likely end up in the local minimum, while the global minimum would rank much better in your preferences. Adding some form of controlled chaos to the algorithm is an easy way to increase the chance of reaching the global minimum, even in much more complex setups than this simple curve.

For a relatively broad class of problems, like selecting the best position of nodes to minimize the length of edges when doing a bitmap representation of a graph structure, an algorithm which works quite well and is simple to code is the simulated annealing algorithm, which works by doing local optimizations, but having a global temperature which adds chaos (the higher the temperature, the more random is the process). The temperature itself decreases with the process, and ultimately reaches 0 (pure lawful optimization).

Such methods are of course "dirty hacks", that are used only when the problem is too complicated and we don't have a purely lawful algorithm that gives the answer, or (most of the time) when we do have one, but with an exponential complexity, meaning we can't run it in real life.

The same idea applies to human creativity : chaos wouldn't be needed, nor useful, if we had a fully working algorithm to write the best books or songs or make the best paintings. But since we don't, using a purely lawful process has a risk (but yes, only a risk) of getting us stuck into a local minimum - improving the methods of the previous generation of artists, but not inventing brand new styles of art. This is a similar concept to the "jumping out of the system" described by Douglas Hofstadter and analyzed by Eliezer. JOOTSing is escaping a local minimum. It's escaping the safe warmth of the valley, climbing the cold and dangerous mountain top, to find another, more fertile valley on the other side. That requires to violate the rules of "staying into the safe and warm valley".

(Note : there is somehow an analogy between the use of drugs and the simulated annealing : drugs induce a state of high chaos, which then slowly goes down as the drug effects disappears. Or at least I was told so, since I never tried personally. But that seems a surface analogy to me, so I won't give it much credit).

Inhibitions and art

Or another way to consider it is to look at is inhibitions : the human mind contains a process that'll check your actions (painting and writing in that case, but applies more broadly) and sometimes say "no, don't do that, you'll look as a fool". Those inhibitions are usually here to protect us from botching in social situations. But they are (as most of the human brain) imperfectly calibrated, and will tend to repress anything that goes out of the current norm. Lowering those inhibitions increases the risk of botching - but also the chance of doing something awesome.

This points to much more general pattern, which applies when what matters is not improving your average gain, but your chance of being one of the few best. Consider you've a task to do, and two ways to achieve it. Way A is quite classical, and doesn't involve much risk. Way B is much less proven, and involves much risk of doing both better and worse. Being a role player, I usually use dice rolls to model those kinds of process. Let's say process A is 20d10. That means, rolling 20 times a 10-sided die, and doing the sum. This will give an expected value of 110, with only 1% of the rolls above 140. Now process B is 2d100 (rolling 2 times a 100-sided die and doing the sum). This will give a lower expected value, of 101 instead of 110. But with 18% of the rolls above 140. Here is a picture of the two process (way A in green, way B in red) generated with a quick Python script :

If what matters is doing your best in average (your score at the task will directly map to an amount of money between $2 and $200), then the best choice is to look only at the expected value of A and B, and select the one which has the best expected value, so A in this case, as you can see, the green curve peaks at a higher value.

But if what matters is not doing the best in average, but being the best : 100 people are performing the task, and the best will take the prize, the rest won't have anything. Then, you except one of the 100 to be above 140, even if they all use way A. So for yourself, if you use way A, you only have 1 chance in 100 to be above 140. If you use way B, you've 18 chances in 100 of beating the 140 mark. Looking at the curve, there is much bigger blob of the red curve that goes to very high values.

When looking at arts, we don't regard the average. Countless people write books or paint. Almost everyone at least tried once. What history remembers are the few best of their time. Not those who did better in average, but those who manage to do better than most of their peers. Those to the right of the picture, in which the amplitude of the green curve is nearly void, but the red curve still exists.

The complexity of testing certain hypothesis

I emitted many hypothesis in this article, to try to explain the common knowledge that "so many great artists take drugs", and more generally to look into the reasons for which chaos can, in some cases, improve a result.

All those hypothesis seem totally plausible to me - and I would say that they all play a role in the process. But saying "everything plays a role" is not saying much, a graph with all possible edges contains as much informations as a graph with no edge. What would be require now is to consider how much each hypothesis contributes to the result - and then, probably one or two will account for most.

But how can we setup such a test ? In physics, doing experiments is relatively easy. It can costs a lot like building the LHC or sending the Hubble space telescope in orbit, but still, devising experiments is relatively easy. In social sciences, it's often much harder. Most social science experiments are done on a panel of test subjects (with a control group, ...). But right now we are speaking of the best artists. How can we build such a panel ? Defining who are the best artists is a very hard task. And then, getting them to participate in studies...

The simplest hypothesis to test, the availability bias, would require a procedure like (numbers can be adjusted) :

1. Take 1000 people at random, from various ages, social classes and background.
2. Ask for each of them to name the 10 artists they like the most (without of course mentioning the purpose of the experiment).
3. For each artist nominated by at least 4 persons, look if that artist did take drugs.
4. Compare with the average drug use.

But even that is not without troubles : for 3., how can you be sure an artist didn't take drug secretly, especially in time/places where drug use is prohibited or frown upon ? For 4., how do you ponder for the variation in drug use depending of the place/time ?

Does anyone know of such a study (I couldn't find any, but I'm not well versed in the art for looking for social science studies) ?

For the other hypothesis, testing them becomes even harder.

Conclusion

As Eliezer explained, pure chaos cannot lead to anything but static on a TV screen. Any optimization process, and art is one, requires a lawful part. But as I showed, for several reasons, an imperfect optimization process may perform better with a limited amount of added chaos. Since the human brain is an imperfect optimization process, it would not be surprising that in the purpose of creating awesome pieces of art in a limited time, some added chaos can help. But on the other hand, there are other reasons for which there could be a common knowledge that "artistic creativity requires some chaos" even if it were not true. And it is very hard to tell apart the various reasons.

But even if some amount of chaos can help in generating exceptionally awesome pieces of art, it should not shadow the fact that the lawful part of process is absolutely required, and even the most important one; nor that chaos can only be useful when the optimization is itself imperfect. Improving the quality of the optimization process (by, for example, raising the sanity waterline or understanding better the human brain) would lower the need of chaos to generate the same awesomeness.

PS : I post that to "Less Wrong discussion", for initial review and because it's half-way between a "real" article and a call for discussion on the topic. Depending of feedback, I hope to repost it to "main Less Wrong", hopefully improved from the feedback.