Filter Last three months

Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Bring up Genius

45 Viliam 08 June 2017 05:44PM

(This is a "Pareto translation" of Bring up Genius by László Polgár, the book recently mentioned at Slate Star Codex. I hope that selected 20% of the book text, translated approximately, could still convey 80% of its value, while taking an order of magnitude less time and work than a full and precise translation. The original book is written in an interview form, with questions and answers; to keep it short, I am rewriting it as a monologue. I am also taking liberty of making many other changes in style, and skipping entire parts, because I am optimizing for my time. Instead of the Hungarian original, I am using an Esperanto translation Eduku geniulon as my source, because that is the language I am more fluent in.)


Genius = work + luck

This is my book written in 1989 about 15 years of pedagogic experiment with my daughters. It is neither a recipe, nor a challenge, just a demonstration that it is possible to bring up a genius intentionally.

The so-called miracle children are natural phenomena, created by their parents and society. Sadly, many potential geniuses disappear without anyone noticing the opportunity, including themselves.

Many people in history did a similar thing by accident; we only repeated it on purpose.

1. Secrets of the pedagogic experiment

1.1. The Polgár family

The Polgár sisters (Susan, Sofia, Judit) are internationally as famous as Rubik Ernő, the inventor of the Rubik Cube.

Are they merely their father's puppets, manipulated like chess figures? Hardly. This level of success requires agency and active cooperation. Puppets don't become geniuses. Contrariwise, I provided them opportunity, freedom, and support. They made most of the decisions.

You know what really creates puppets? The traditional school system. Watch how kids, eagerly entering school in September, mostly become burned out by Christmas.

Not all geniuses are happy. Some are rejected by their environment, or they fail to achieve their goals. But some geniuses are happy, accepted by their environment, succeed, and contribute positively to the society. I think geniuses have a greater chance to be happy in life, and luckily my daughters are an example of that.

I was a member of the Communist Party for over ten years, but I disagreed with many things; specifically the lack of democracy, and the opposition to elite education.

I work about 15 hours a day since I was a teenager. I am obsessed with high quality. Some people say I am stubborn, even aggressive. I am trying hard to achieve my goals, and I experienced a lot of frustration; seems to me some people were trying to destroy us. We were threatened by high-ranking politicians. We were not allowed to travel abroad until 1985, when Susan was already the #1 in international ranking of female chess players.

But I am happy that I have a great family, happy marriage, three successful children, and my creative work has an ongoing impact.

1.2 Nature or nurture?

I believe that any biologically healthy child can be brought up to a genius. Me and my wife have read tons of books and studies. Researching the childhoods of many famous people that they all specialized early, and each of them had a strongly supportive parent or teacher or trainer. We concluded: geniuses are not born; they are made. We proved that experimentally. We hope that someone will build a coherent pedagogical system based on our hypothesis.

Most of what we know about genetics [as of 1989] is about diseases. Healthy brains are flexible. Education was considered important by Watson and Adler. But Watson never actually received the "dozen healthy infants" to bring up, so I was the first one to do this experiment. These are my five principles:

* Human personality is an outcome of the following three: the gifts of nature, the support of environment, and the work of one's own. Their relative importance depends on age: biology is strongest with the newborn, society with the ten years old, and later the importance of one's own actions grows.

* There are two aspects of social influence: the family, and the culture. Humans are naturally social, so education should treat the child as a co-author of themselves.

* I believe that any healthy child has sufficient general ability, and can specialize in any type of activity. Here I differ from the opinion of many teachers and parents who believe that the role of education is to find a hidden talent in the child. I believe that the child has a general ability, and achieves special skills by education.

* The development of the genius needs to be intentionally organized; it will not happen at random.

* People should strive for maximum possible self-realization; that brings happiness both to them and to the people around them. Pedagogy should not aim for average, but for excellence.

2. A different education

2.1. About contemporary schools

We homeschooled our children. Today's schools set a very low bar, and are intolerant towards people different from the average by their talent or otherwise. They don't prepare for real life; don't make kids love learning; don't instigate greater goals; bring up neither autonomous individuals nor collectives.

Which is an unsurprising outcome, if you only have one type of school, each school containing a few exceptional kids among many average ones and a few feeble ones. Even the average ones are closer to the feeble ones that to the exceptional ones. And the teacher, by necessity, adapts to the majority. There is not enough space for individual approach, but there is a lot of mindless repetition. Sure, people talk a lot about teaching problem-solving skills, but that never happens. Both the teachers and the students suffer at school.

The gifted children are bored, and even tired, because boredom is more tedious than appropriate effort. The gifted children are disliked, just like everyone who differs from the norm. Many gifted children acquire psycho-somatic problems, such as insomnia, headache, stomach pain, neuroses. Famous people often had trouble at school; they were considered stupid and untalented. There is bullying, and general lack of kindness. There are schools for gifted children in USA and USSR, but somehow not in Hungary [as of 1989].

I had to fight a lot to have my first daughter home-schooled. I was afraid school would endanger the development of her abilities. We had support of many people, including pedagogues, but various bureaucrats repeatedly rejected us, sometimes with threats. Finally we received an exceptional permission by the government, but it only applied for one child. So with the second daughter we had to go through the same process again.

2.2. Each child is a promise

It is crucial to awaken and keep the child's interest, convince them that the success is achievable, trust them, and praise them. When the child likes the work, it will work fruitfully for long time periods. A profound interest develops personality and skills. A motivated child will achieve more, and get tired less.

I believe in positive motivation. Create a situation where many successes are possible. Successes make children confident; failures make them insecure. Experience of success and admiration by others motivates and accelerates learning. Failure, fear, and shyness decrease the desire to achieve. Successes in one field even increase confidence in other fields.

Too much praise can cause overconfidence, but it is generally safer to err on the side of praising more rather than less. However, the praise must be connected to a real outcome.

Discipline, especially internal psychological, also increases skills.

I believe the age between 3 and 6 years is very important, and very underestimated. No, those children are not too young to learn. Actually, that's when their brains are evolving the most. They should learn foreign languages. In multilingual environments children do that naturally.

Play is important for children, but play is not an opposite of work. Gathering information and solving problems is fun. Provide meaningful activities, instead of compartmentalized games. A game without learning is merely a surrogate activity. Gifted children prefer games that require mental activity. There is a continuum between learning and playing (just like between work and hobby for adults). Brains, just like muscles, becomes stronger by everyday activity.

My daughters used intense methods to learn languages; and chess; and table tennis. Is there a risk of damaging their personality by doing so? Maybe, but I believe the risks of damaging the personality by spending six childhood years without any effort are actually greater.

When my daughters were 15, 9, 8 years old, we participated in a 24-hour chess tournament, where you had to play 100 games in 24 hours. (Most participants were between age 25 and 30.) Susan won. The success rates during the second half of the tournament were similar to those during the first half of the tournament, for all three girls, which shows that children are capable of staying focused for long periods of time. But this was an exceptional load.

2.3. Genius - a gift or a curse?

I am not saying that we should bring up each child as a genius; only that bringing up children as geniuses is possible. I oppose uniform education, even a hypothetical one that would use my methods.

Public ideas of geniuses is usually one of two extremes. Either they are all supposed to be weird and half-insane; or they are all supposed to be CEOs and movie stars. Psychology has already moved beyond this. They examined Einstein's brain, but found no difference in weight or volume compared with an average person. For me, genius is an average person who has achieved their full potential. Many famous geniuses attribute their success to hard work, discipline, attention, love of work, patience, time.

All healthy newborns are potential geniuses, but whether they become actual geniuses, depends on their environment, education, and their own effort. For example, in the 20th century more people became geniuses than in the 19th or 18th century, inter alia because of social changes. Geniuses need to be liberated. Hopefully in the future, more people will be free and fully developed, so being a genius will become a norm, not an exception. But for now, there are only a few people like that. As people grow up, they lose the potential to become geniuses. I estimate that an average person's chance to become a genius is about 80% at age 1; 60% at age 3; 50% at age 6; 40% at age 12; 30% at age 16; 20% at age 18; only 5% at age 20. Afterwards it drops to a fraction of percent.

A genius child can surpass their peers by 5 or 7 years. And if a "miracle child" doesn't become a "miracle adult", I am convinced that their environment did not allow them to. People say some children are faster and some are slower; I say they don't grow up in the same conditions. Good conditions allow one to progress faster. But some philosophers or writers became geniuses at old age.

People find it difficult to accept those who differ from the average. Even some scientists; for example Einstein's theory of relativity was opposed by many. My daughters are attacked not just by public opinion, but also by fellow chess players.

Some geniuses are unhappy about their situation. But many enjoy the creativity, perceived beauty, and success. Geniuses can harm themselves by having unrealistic expectations of their goals. But most of the harm comes from outside, as a dismissal of their work, or lack of material and moral support, baseless criticism. Nowadays, one demagogue can use the mass communication media to poison the whole population with rage against the representatives of national culture.

As the international communication and exchange of ideas grows, geniuses become more important than ever before. Education is necessary to overcome economical problems; new inventions create new jobs. But a genius provokes the anger of people, not by his behavior, but by his skills.

2.4. Should every child become a celebrity?

I believe in diversity in education. I am not criticizing teachers for not doing things my way. There are many other attempts to improve education. But I think it is now possible to aim even higher, to bring up geniuses. I can imagine the following environments where this could be done:

* Homeschooling, i.e. teaching your biological or adopted children. Multiple families could cooperate and share their skills.

* Specialized educational facility for geniuses; a college or a family-type institution.

Homeschooling, or private education with parental oversight, are the ancient methods for bringing up geniuses. Families should get more involved in education; you can't simply outsource everything to a school. We should support families willing to take an active role. Education works better in a loving environment.

Instead of trying to a find a talent, develop one. Start specializing early, at the age of 3 or 4. One cannot become an expert on everything.

My daughters played chess 5 or 6 hours a day since their age of 4 or 5. Similarly, if you want ot become a musician, spend 5 or 6 hours a day doing music; if a physicist, do physics; if a linguist, do languages. With such intense instruction, the child will soon feel the knowledge, experience success, and soon becomes able to use this knowledge independently. For example, after learning Esperanto 5 or 6 hours a day for a few months, the child can start corresponding with children from other countries, participate at international meet-ups, and experience the conversations in a foreign language. That is at the same time pleasant, useful for the child, and useful for the society. The next year, start with English, then German, etc. Now the child enjoys this, because it obviously makes sense. (Unlike at school, where most learning feels purposeless.) In chess, the first year makes you an average player, three years a great player, six years a master, fifteen years a grandmaster. When a 10-years old child surpasses an average adult at some skill, it is highly motivating.

Gifted children need financial support, to cover the costs of books, education, and travel.

Some people express concern that early specialization may lead to ignorance of everything else. But it's the other way round; abilities formed in one area can transfer to other areas. One learns how to learn.

Also, the specialization is relative. If you want to become e.g. a computer programmer, you will learn maths, informatics, foreign languages; when you become famous, you will travel, meet interesting people, experience different cultures. My daughters, in addition to being chess geniuses, speak many foreign languages, travel, do sports, write books, etc. Having deep knowledge about something doesn't imply ignorance about everything else. On the other hand, a misguided attempt to become an universalist can result in knowing nothing, in mere pretend-knowledge of everything.

Emotional and moral education must do together with the early specialization, to develop a complex personality. We wanted our children to be enthusiastic, courageous, persistent, to be objective judges of things and people, to resist failure and avoid temptations of success, to handle frustration and tolerate criticism even when it is wrong, to make plans, to manage their emotions. Also, to love and respect people, and to prefer creative work to physical pleasure or status symbols. We told them that they can achieve greatness, but that there can be only one world champion, so their goal should rather be to become good chess players, be good at sport, and be honest people.

Pedagogy puts great emphasis on being with children of the same age. I think that mental peers are more important than age peers. It would harm a gifted child to be forced to spend most of their time exclusively among children of the same age. On the other hand, spending most of the time with adults brings the risk that the child will learn to rely on them all the time, losing independence and initiative. You need to find a balance. I believe the best company would be of similar intellectual level, similar hobbies, and good relations.

For example, if Susan at 13 years old would be forced to play chess exclusively with 13 years old children, it would harm both sides. She could not learn anything from them; they would resent losing constantly.

Originally, I hoped I could bring up each daughter as a genius in a different field (e.g. mathematics, chess, music). It would be a more convincing evidence that you can bring up a genius of any kind. And I believe I would have succeeded, but I was constrained by money and time. We would need three private teachers, would have to go each day to three different places, would have to buy books for maths and chess and music (and the music instruments). By making them one team, things became easier, and the family has more things in common. Some psychologists worried that children could be jealous of each other, and hate each other. But we brought them up properly, and this did not happen.

This is how I imagine a typical day at a school for geniuses:

* 4 hours studying the subject of specialization, e.g. chess;

* 1 hour studying a foreign language; Esperanto at the first year, English at the second, later choose freely; during the first three months this would increase to 3 hours a day (by reducing the subject of specialization temporarily); traveling abroad during the summer;

* 1 hour computer science;

* 1 hour ethics, psychology, pedagogy, social skills;

* 1 hour physical education, specific form chosen individually.

Would I like to teach at such school? In theory yes, but in practice I am already burned out from the endless debates with authorities, the press, opinionated pedagogues and psychologists. I am really tired of that. The teachers in such school need to be protected from all this, so they can fully focus on their work.

2.5. Esperanto: the first step in learning foreign languages

Our whole family speaks Esperanto. It is a part of our moral system, a tool for equality of people. There are many prejudices against it, but the same was true about all progressive ideas. Some people argue by Bible that multiple languages are God's punishment we have to endure. Some people invested many resources into learning 2 or 3 or 4 foreign languages, and don't want to lose the gained position. Economically strong nations enforce their own languages as part of dominance, and the speakers of other languages are discriminated against. Using Esperanto as everyone's second language would make the international communication more easy and egalitarian. But considering today's economical pressures, it makes sense to learn English or Russian or Chinese next.

Esperanto has a regular grammar with simple syntax. It also uses many Latin, Germanic, and Slavic roots, so as a European, even if you are not familiar with the language, you will probably recognize many words in a text. This is an advantage from pedagogical point of view: you can more easily learn its vocabulary and its grammar; you can learn the whole language about 10 times easier than other languages.

It makes a great example of the concept of a foreign language, which pays off when learning other languages later. It is known that having learned one foreign language makes learning another foreign language easier. So, if learning Esperanto takes 10 times less time than learning another language, such as English, then if already knowing another foreign language makes learning the second one at least 10% more efficient, it makes sense to learn Esperanto first. Also, Esperanto would be a great first experience for students who have difficulty learning languages; they would achieve success faster.

3. Chess

3.1. Why chess?

Originally, we were deciding between mathematics, chess, and foreign languages. Finally we chose chess, because the results in that area are easy to measure, using a traditional and objective system, which makes it easier to prove whether the experiment succeeded or failed. Which was a lucky choice in hindsight, because back then we had no idea how many obstacles we will have to face. If we wouldn't be able to prove our results unambiguously, the attacks against us would have been much stronger.

Chess seemed sufficiently complex (it is a game, a science, an art, and a sport at the same time), so the risks of overspecialization were smaller; even if children would later decide they are tired of chess, they would keep some transferable skills. And the fact that our children were girls was a bonus: we were able to also prove that girls can be as intellectually able as boys; but for this purpose we needed an indisputable proof. (Although, people try to discount this proof anyway, saying things like: "Well, chess is simple, but try doing the same in languages, mathematics, or music!")

The scientific aspect of chess is that you have to follow the rules, analyze the situation, apply your intuition. If you have a favorite hypothesis, for example a favorite opening, but you keep losing, you have to change your mind. There is an aesthetic dimension in chess; some games are published and enjoyed not just because of their impressive logic, but because they are beautiful in some sense, they do something unexpected. And most people are not familiar with this chess requires great physical health. All the best chess players do some sport, and it is not a coincidence. Also it is organized similarly to sports: it has tournaments, players, spectators; you have to deal with the pain of losing, you have to play fair, etc.

3.2. How did the Polgár sisters start learning chess?

I don't have a "one weird trick" to teach children chess; it's just my general pedagogical approach, applied to chess. Teach the chess with love, playfully. Don't push it too forcefully. Remember to let the child win most of the time. Explain to the child that things can be learned, and that this also applies to chess. Don't worry if the child keeps jumping during the game; it could be still thinking about the game. Don't explain everything; provide the child an opportunity to discover some things independently. Don't criticize failure, praise success.

Start with shorter lessons, only 30 minutes and then have a break. Start by solving simple problems. Our girls loved the "checkmate in two/three moves" puzzles. Let the child play against equally skilled opponents often. For a child, it is better to play many quick games (e.g. with 5-minute timers), than a few long ones. Participate in tournaments appropriate for the child's current skill.

We have a large library of different games. They are indexed by strategy, and by names of players. So the girls can research their opponent's play before the tournament.

When a child loses the tournament, don't criticize them; the child is already sad. Offer support; help them analyze the mistakes.

When my girls write articles about chess, it makes them think deeply about the issue.

All three parts of the game opening, middle game, ending require same amount of focus. Some people focus too much on the endings, and neglect the rest. But at tournament, a bad opening can ruin the whole game.

Susan had the most difficult situation of the three daughters. In hindsight, having her learn 7 or 8 foreign languages was probably too much; some of that time would be better spent further improving her chess skills. As the oldest one, she also faced the worst criticism from haters; as a consequence she became the most defensive player of them. The two younger sister had the advantage that they could oppose the same pressures together. But still, I am sure that without those pressures, they also could have progressed even faster.

Politicians influenced the decisions of the Hungarian Chess Association; as a result my daughters were often forbidden from participation at international youth competitions, despite being the best national players. They wanted to prevent Susan from becoming the worldwide #1 female chess player. Once they even "donated" 100 points to her competitor, to keep Susan at the 2nd place. Later they didn't allow her to participate in the international male tournaments, although her results in the Hungarian male tournaments qualified her for that. The government regularly refused to issue passports to us, claiming that "our foreign travels hurt the public order". Also, it was difficult to find a trainer for my daughters, despite them being at the top of world rankings. Only recently we received a foreign help; a patron from Netherlands offered to pay trainers and sparring partners for my daughters, and also bought Susan a personal computer. A German journalist gave us a program and a database, and taught children how to use it.

The Hungarian press kept attacking us, published fake facts. We filed a few lawsuits, and won them all, but it just distracted us from our work. The foreign press whether writing from the chess, psychological, or pedagogical perspectives was fair to us; they wrote almost 40 000 articles about us, so finally even the Hungarian chess players, psychologists and pedagogues could learn about us from them.

At the beginning, I was a father, a trainer, and a manager to my daughters. But I am completely underqualified to be their trainer these days, so I just manage their trainers.

Until recently no one believed women could play chess on level comparable with men. Now the three girls together have about 40 Guiness records; they repeatedly outperformed their former records. In a 1988 interview Karpov said: "Susan is extraordinarily strong, but Judit... at such age, neither me nor Kasparov could play like Judit plays."

3.3. How can we make our children like chess?

Some tips for teaching chess to 4 or 5 years old children. First, I made a blank square divided into 8x8 little squares, with named rows and columns. I named a square, my daughter had to find it; then she named a square and I had to find it. Then we used the black-and-white version, and we were guessing the color of the named square without looking.

Then we introduced kings, in a "king vs king" combat; the task was to reach the opposing row of the board with your king. Then we added a pawn; the goal remained to reach the opposing row. After a month of playing, we introduced the queen, and the concept of checkmate. Later we gradually added the remaining pieces (knights were the most difficult).

Then we solved about thousand "checkmate in one move" puzzles. Then two moves, three moves, four moves. That took another 3 or 4 months. And only afterwards we started really playing against each other.

To provide an advantage for the child, don't play with less pieces, because that changes the structure of the game. Instead, provide yourself a very short time limit, or deliberately make a mistake, so the child can learn to notice them.

Have patience, if some phase takes a lot of time. On stronger fundamentals, you can later build better. This is where I think our educational system makes great mistakes. Schools don't teach intensely, so children keep forgetting most of what they learned during the long spaces between the lessons. And then, despite not having fully mastered the first step, they move to the second one, etc.

3.4. Chess and psychology

Competitive chess helps develop personality: will, emotion, perseverance, self-discipline, focus, self-control. It develops intellectual skills: memory, combination skills, logic, proper use of intuition. Understanding your opponent's weakness will help you.

People overestimate how much IQ tests determine talent. Measurements of people talented in different areas show that their average is only a bit above the average of the population.

3.5. Emancipation of women

Some people say, incorrectly, that my daughter won the male chess championship. But there is officially no such thing as "male chess championship", there is simply chess championship, open to both men and women. (And then, there is a separate female chess championship, only for women, but that is considered second league.)

I prepared the plan for my children before they were born. I didn't know I would have all girls, so I did not expect this special problem: the discrimination of women. I wanted to bring up my daughter Susan exactly according to the plan, but many people tried to prevent it; they insisted that she cannot compete with boys, that she should only compete with girls. Thus my original goal of proving that you can bring up a genius, became indirectly a goal of proving that there are no essential intellectual differences between men and women, and therefore one can't use that argument as an excuse for subjugation of women.

People kept telling me that I can only bring up Susan to be a female champion, not to compete with men. But I knew that during elementary school, girls can compete with boys. Only later, when they start playing the female role, when they are taught to clean the house, wash laundry, cook, follow the fashion, pay attention to details of clothing, and try getting married as soon as possible when they are expected to do other things than boys are expected to do that has a negative impact on developing their skills. But family duties and bringing up children can be done by both parents together.

Women can achieve same results, if they can get similar conditions. I tried to do that for my daughters, but I couldn't convince the whole society to treat them the same.

We know about differences between adult men and women, but we don't know whether they were caused by biology or education. And we know than e.g. in mathematics and languages, during elementary and high schools girls progress at the same pace as boys, and only later the differences appear. This is an evidence in favor of equality. We do not know what children growing up without discrimination would be like.

On the other hand, the current system also provides some advantages for women; for example the female chess players don't need to work that hard to become the (female) elite, and some of them don't want to give that up. Such women are among the greatest opponents of my daughters.

4. The meaning of this whole affair

4.1. Family value

I am certain that without a good family background the success of my daughters would not be possible. It is important, before people marry, to have a clear idea of what expect from their marriage. When partners cooperate, the mutual help, the shared experiences, education of children, good habits, etc. can deepen their love. Children need family without conflicts to feel safe. But of course, if the situation becomes too bad, the divorce might become the way to reduce conflicts.

To bring up a genius, it is desirable for one parent to stay at home and take care of children. But it can be the father, too.

[Klára Polgár says:] When I met László, my first impression was that he was an interesting person full of ideas, but one should not believe even half of them.

When Susan was three and half, László said it was time for her to specialize. She was good at math; at the age of four she already learned the material of the first four grades. Once she found chess figures in the box, and started playing with them as toys. László was spending a lot of time with her, and one day I was surprised to see them playing chess. László loved chess, but I never learned it.

So, we could have chosen math or foreign languages, but we felt that Susan was really happy playing chess, and she started being good at it. But our parents and neighbors shook their heads: "Chess? For a girl?" People told me: "What kind of a mother are you? Why do you allow your husband to play chess with Susan?" I had my doubts, but now I believe I made the right choice.

People are concerned whether my children had real childhood. I think they are at least as happy as their peers, probably more.

I always wanted to have a good, peaceful family life, and I believe I have achieved that. [End of Klára's part.]

4.2. Being a minority

It is generally known that Jewish people achieved many excellent results in intellectual fields. Some ask whether the cause of this is biologic or social. I believe it is social.

First, Jewish families are usually traditional, stable, and care a lot about education. They knew that they will be discriminated against, and will have to work twice as hard, and that at any moment they may be forced to leave their home, or even country, so their knowledge might be the only thing they will always be able to keep. Jewish religion requires parents to educate their children since early childhood; Talmud requires parents to become the child's first teachers.

4.3. Witnesses of the genius education: the happy children

I care about happiness of my children. But not only I want to make them happy, I also want to develop their ability to be happy. And I think that being a genius is the most certain way. The life of a genius may be difficult, but happy anyway. On the other hand, average people, despite seemingly playing it safe, often become alcoholics, drug addicts, neurotics, loners, etc.

Some geniuses become unhappy with their profession. But even then I believe it is easier for a genius to change professions.

Happiness = work + love + freedom + luck

People worry whether child geniuses don't lose their childhood. But the average childhood is actually not as great as people describe it; many people do not have a happy childhood. Parents want to make their children happy, but they often do it wrong: they buy them expensive toys, but they don't prepare them for life; they outsource that responsibility to school, which generally does not have the right conditions.

And when parents try to fully develop the capabilities of their children, instead of social support they usually get criticism. People will blame them for being overly ambitious, for pushing the children to achieve things they themselves failed at. I personally know people who tried to educate their children similarly to how we did, but the press launched a full-scale attack against them, and they gave up.

My daughters' lives are full of variety. They have met famous people: presidents, prime ministers, ambassadors, princess Diana, millionaires, mayors, UN delegates, famous artists, other olympic winners. They appeared in television, radio, newspapers. They traveled around the whole world; visited dozens of famous places. They have hobbies. They have friends in many parts of the world. And our house is always open to guests.

4.4. Make your life an ethical model

People reading this text may be surprised that they expected a rational explanation, while I mention emotions and morality a lot. But those are necessary for good life. Everyone should try to improve themselves in these aspects. The reason why I did not give up, despite all the obstacles and malice, is because for me, to live morally and create good, is an internal law. I couldn't do otherwise. I already know that even writing this very book will initiate more attacks, but I am doing it regardless.

And morality is also a thing we are not born with, but which needs to be taught to us, preferably in infancy. And we need to think about it, instead of expecting it to just happen. And the schools fail in this, too. I see it as an integral part of bringing up a genius.

One should aim to be a paragon; to live in a way that will make others want to follow you. Learn and work a lot; expect a lot from yourself and from others. Give love, and receive love. Live in peace with yourself and your neighbors. Work hard to be happy, and to make other people happy. Be a humanist, fight against prejudice. Protect the peace of the family, bring up your children towards perfection. Be honest. Respect freedom of yourself and of the others. Trust humanity; support the communities small and large. Etc.

(The book finishes by listing the achievements of the Polgár sisters, and by their various photos: playing chess, doing sports. I'll simply link their Wikipedia pages: Susan, Sofia, Judit. I hope you enjoyed reading this experimental translation; and if you think I omitted something important, feel free to add the missing parts in the comments. Note: I do believe that this book is generally correct and useful, but that doesn't mean I necessarily agree with every single detail. The opinions expressed here belong to the author; of course, unless some of them got impaired by my hasty translation.)

Dragon Army: Theory & Charter (30min read)

40 Duncan_Sabien 25 May 2017 09:07PM

Author's note: This IS a rationality post (specifically, theorizing on group rationality and autocracy/authoritarianism), but the content is quite cunningly disguised beneath a lot of meandering about the surface details of a group house charter.  If you're not at least hypothetically interested in reading about the workings of an unusual group house full of rationalists in Berkeley, you can stop here.  

Section 0 of 3: Preamble

Purpose of post:  Threefold.  First, a lot of rationalists live in group houses, and I believe I have some interesting models and perspectives, and I want to make my thinking available to anyone else who's interested in skimming through it for Things To Steal.  Second, since my initial proposal to found a house, I've noticed a significant amount of well-meaning pushback and concern à la have you noticed the skulls? and it's entirely unfair for me to expect that to stop unless I make my skull-noticing evident.  Third, some nonzero number of humans are gonna need to sign the final version of this charter if the house is to come into existence, and it has to be viewable somewhere.  I figured the best place was somewhere that impartial clear thinkers could weigh in (flattery).

What is Dragon Army [Barracks]?  It's a high-commitment, high-standards, high-investment group house model with centralized leadership and an up-or-out participation norm, designed to a) improve its members and b) actually accomplish medium-to-large scale tasks requiring long-term coordination.  Tongue-in-cheek referred to as the "fascist/authoritarian take on rationalist housing," which has no doubt contributed to my being vulnerable to strawmanning but was nevertheless the correct joke to be making, lest people misunderstand what they were signing up for.  Aesthetically modeled after Dragon Army from Ender's Game (not HPMOR), with a touch of Paper Street Soap Company thrown in, with Duncan Sabien in the role of Ender/Tyler and Eli Tyre in the role of Bean/The Narrator.

Why?  Current group housing/attempts at group rationality and community-supported leveling up seem to me to be falling short in a number of ways.  First, there's not enough stuff actually happening in them (i.e. to the extent people are growing and improving and accomplishing ambitious projects, it's largely within their professional orgs or fueled by unusually agenty individuals, and not by leveraging the low-hanging fruit available in our house environments).  Second, even the group houses seem to be plagued by the same sense of unanchored abandoned loneliness that's hitting the rationalist community specifically and the millennial generation more generally.  There are a bunch of competitors for "third," but for now we can leave it at that.

"You are who you practice being."

Section 1 of 3: Underlying models

The following will be meandering and long-winded; apologies in advance.  In short, both the house's proposed aesthetic and the impulse to found it in the first place were not well-reasoned from first principles—rather, they emerged from a set of System 1 intuitions which have proven sound/trustworthy in multiple arenas and which are based on experience in a variety of domains.  This section is an attempt to unpack and explain those intuitions post-hoc, by holding plausible explanations up against felt senses and checking to see what resonates.

Problem 1: Pendulums

This one's first because it informs and underlies a lot of my other assumptions.  Essentially, the claim here is that most social progress can be modeled as a pendulum oscillating decreasingly far from an ideal.  The society is "stuck" at one point, realizes that there's something wrong about that point (e.g. that maybe we shouldn't be forcing people to live out their entire lives in marriages that they entered into with imperfect information when they were like sixteen), and then moves to correct that specific problem, often breaking some other Chesterton's fence in the process.

For example, my experience leads me to put a lot of confidence behind the claim that we've traded "a lot of people trapped in marriages that are net bad for them" for "a lot of people who never reap the benefits of what would've been a strongly net-positive marriage, because it ended too easily too early on."  The latter problem is clearly smaller, and is probably a better problem to have as an individual, but it's nevertheless clear (to me, anyway) that the loosening of the absoluteness of marriage had negative effects in addition to its positive ones.

Proposed solution: Rather than choosing between absolutes, integrate.  For example, I have two close colleagues/allies who share millennials' default skepticism of lifelong marriage, but they also are skeptical that a commitment-free lifestyle is costlessly good.  So they've decided to do handfasting, in which they're fully committed for a year and a day at a time, and there's a known period of time for asking the question "should we stick together for another round?"

In this way, I posit, you can get the strengths of the old socially evolved norm which stood the test of time, while also avoiding the majority of its known failure modes.  Sort of like building a gate into the Chesterton's fence, instead of knocking it down—do the old thing in time-boxed iterations with regular strategic check-ins, rather than assuming you can invent a new thing from whole cloth.

Caveat/skull: Of course, the assumption here is that the Old Way Of Doing Things is not a slippery slope trap, and that you can in fact avoid the failure modes simply by trying.  And there are plenty of examples of that not working, which is why Taking Time-Boxed Experiments And Strategic Check-Ins Seriously is a must.  In particular, when attempting to strike such a balance, all parties must have common knowledge agreement about which side of the ideal to err toward (e.g. innocents in prison, or guilty parties walking free?).


Problem 2: The Unpleasant Valley

As far as I can tell, it's pretty uncontroversial to claim that humans are systems with a lot of inertia.  Status quo bias is well researched, past behavior is the best predictor of future behavior, most people fail at resolutions, etc.

I have some unqualified speculation regarding what's going on under the hood.  For one, I suspect that you'll often find humans behaving pretty much as an effort- and energy-conserving algorithm would behave.  People have optimized their most known and familiar processes at least somewhat, which means that it requires less oomph to just keep doing what you're doing than to cobble together a new system.  For another, I think hyperbolic discounting gets way too little credit/attention, and is a major factor in knocking people off the wagon when they're trying to forego local behaviors that are known to be intrinsically rewarding for local behaviors that add up to long-term cumulative gain.

But in short, I think the picture of "I'm going to try something new, eh?" often looks like this:

... with an "unpleasant valley" some time after the start point.  Think about the cold feet you get after the "honeymoon period" has worn off, or the desires and opinions of a military recruit in the second week of a six-week boot camp, or the frustration that emerges two months into a new diet/exercise regime, or your second year of being forced to take piano lessons.

The problem is, people never make it to the third year, where they're actually good at piano, and start reaping the benefits, and their System 1 updates to yeah, okay, this is in fact worth it.  Or rather, they sometimes make it, if there are strong supportive structures to get them across the unpleasant valley (e.g. in a military bootcamp, they just ... make you keep going).  But left to our own devices, we'll often get halfway through an experiment and just ... stop, without ever finding out what the far side is actually like.

Proposed solution: Make experiments "unquittable."  The idea here is that (ideally) one would not enter into a new experiment unless a) one were highly confident that one could absorb the costs, if things go badly, and b) one were reasonably confident that there was an Actually Good Thing waiting at the finish line.  If (big if) we take those as a given, then it should be safe to, in essence, "lock oneself in," via any number of commitment mechanisms.  Or, to put it in other words: "Medium-Term Future Me is going to lose perspective and want to give up because of being unable to see past short-term unpleasantness to the juicy, long-term goal?  Fine, then—Medium-Term Future Me doesn't get a vote."  Instead, Post-Experiment Future Me gets the vote, including getting to update heuristics on which-kinds-of-experiments-are-worth-entering.

Caveat/skull: People who are bad at self-modeling end up foolishly locking themselves into things that are higher-cost or lower-EV than they thought, and getting burned; black swans and tail risk ends up making even good bets turn out very very badly; we really should've built in an ejector seat.  This risk can be mostly ameliorated by starting small and giving people a chance to calibrate—you don't make white belts try to punch through concrete blocks, you make them punch soft, pillowy targets first.

And, of course, you do build in an ejector seat.  See next.


Problem 3: Saving Face

If any of you have been to a martial arts academy in the United States, you're probably familiar with the norm whereby a tardy student purchases entry into the class by first doing some pushups.  The standard explanation here is that the student is doing the pushups not as a punishment, but rather as a sign of respect for the instructor, the other students, and the academy as a whole.

I posit that what's actually going on includes that, but is somewhat more subtle/complex.  I think the real benefit of the pushup system is that it closes the loop.  

Imagine you're a ten year old kid, and your parent picked you up late from school, and you're stuck in traffic on your way to the dojo.  You're sitting there, jittering, wondering whether you're going to get yelled at, wondering whether the master or the other students will think you're lazy, imagining stuttering as you try to explain that it wasn't your fault—

Nope, none of that.  Because it's already clearly established that if you fail to show up on time, you do some pushups, and then it's over.  Done.  Finished.  Like somebody sneezed and somebody else said "bless you," and now we can all move on with our lives.  Doing the pushups creates common knowledge around the questions "does this person know what they did wrong?" and "do we still have faith in their core character?"  You take your lumps, everyone sees you taking your lumps, and there's no dangling suspicion that you were just being lazy, or that other people are secretly judging you.  You've paid the price in public, and everyone knows it, and this is a good thing.

Proposed solution: This is a solution without a concrete problem, since I haven't yet actually outlined the specific commitments a Dragon has to make (regarding things like showing up on time, participating in group activities, and making personal progress).  But in essence, the solution is this: you have to build into your system from the beginning a set of ways-to-regain-face.  Ways to hit the ejector seat on an experiment that's going screwy without losing all social standing; ways to absorb the occasional misstep or failure-to-adequately-plan; ways to be less-than-perfect and still maintain the integrity of a system that's geared toward focusing everyone on perfection.  In short, people have to know (and others have to know that they know, and they have to know that others know that they know) exactly how to make amends to the social fabric, in cases where things go awry, so that there's no question about whether they're trying to make amends, or whether that attempt is sufficient.  

Caveat/skull: The obvious problem is people attempting to game the system—they notice that ten pushups is way easier than doing the diligent work required to show up on time 95 times out of 100.  The next obvious problem is that the price is set too low for the group, leaving them to still feel jilted or wronged, and the next obvious problem is that the price is set too high for the individual, leaving them to feel unfairly judged or punished (the fun part is when both of those are true at the same time).  Lastly, there's something in the mix about arbitrariness—what do pushups have to do with lateness, really?  I mean, I get that it's paying some kind of unpleasant cost, but ...

Problem 4: Defections & Compounded Interest

I'm pretty sure everyone's tired of hearing about one-boxing and iterated prisoners' dilemmas, so I'm going to move through this one fairly quickly even though it could be its own whole multipage post.  In essence, the problem is that any rate of tolerance of real defection (i.e. unmitigated by the social loop-closing norms above) ultimately results in the destruction of the system.  Another way to put this is that people underestimate by a couple of orders of magnitude the corrosive impact of their defections—we often convince ourselves that 90% or 99% is good enough, when in fact what's needed is something like 99.99%.

There's something good that happens if you put a little bit of money away with every paycheck, and it vanishes or is severely curtailed once you stop, or start skipping a month here and there.  Similarly, there's something good that happens when a group of people agree to meet in the same place at the same time without fail, and it vanishes or is severely curtailed once one person skips twice.

In my work at the Center for Applied Rationality, I frequently tell my colleagues and volunteers "if you're 95% reliable, that means I can't rely on you."  That's because I'm in a context where "rely" means really trust that it'll get done.  No, really.  No, I don't care what comes up, DID YOU DO THE THING?  And if the answer is "Yeah, 19 times out of 20," then I can't give that person tasks ever again, because we run more than 20 workshops and I can't have one of them catastrophically fail.

(I mean, I could.  It probably wouldn't be the end of the world.  But that's exactly the point—I'm trying to create a pocket universe in which certain things, like "the CFAR workshop will go well," are absolutely reliable, and the "absolute" part is important.)

As far as I can tell, it's hyperbolic discounting all over again—the person who wants to skip out on the meetup sees all of these immediate, local costs to attending, and all of these visceral, large gains to defection, and their S1 doesn't properly weight the impact to those distant, cumulative effects (just like the person who's going to end up with no retirement savings because they wanted those new shoes this month instead of next month).  1.01^n takes a long time to look like it's going anywhere, and in the meantime the quick one-time payoff of 1.1 that you get by knocking everything else down to .99^n looks juicy and delicious and seems justified.

But something magical does accrue when you make the jump from 99% to 100%.  That's when you see teams that truly trust and rely on one another, or marriages built on unshakeable faith (and you see what those teams and partnerships can build, when they can adopt time horizons of years or decades rather than desperately hoping nobody will bail after the third meeting).  It starts with a common knowledge understanding that yes, this is the priority, even—no, wait, especially—when it seems like there are seductively convincing arguments for it to not be.  When you know—not hope, but know—that you will make a local sacrifice for the long-term good, and you know that they will, too, and you all know that you all know this, both about yourselves and about each other.

Proposed solution: Discuss, and then agree upon, and then rigidly and rigorously enforce a norm of perfection in all formal undertakings (and, correspondingly, be more careful and more conservative about which undertakings you officially take on, versus which things you're just casually trying out as an informal experiment), with said norm to be modified/iterated only during predecided strategic check-in points and not on the fly, in the middle of things.  Build a habit of clearly distinguishing targets you're going to hit from targets you'd be happy to hit.  Agree upon and uphold surprisingly high costs for defection, Hofstadter style, recognizing that a cost that feels high enough probably isn't.  Leave people wiggle room as in Problem 3, but define that wiggle room extremely concretely and objectively, so that it's clear in advance when a line is about to be crossed.  Be ridiculously nitpicky and anal about supporting standards that don't seem worth supporting, in the moment, if they're in arenas that you've previously assessed as susceptible to compounding.  Be ruthless about discarding standards during strategic review; if a member of the group says that X or Y or Z is too high-cost for them to sustain, believe them, and make decisions accordingly.

Caveat/skull: Obviously, because we're humans, even people who reflectively endorse such an overall solution will chafe when it comes time for them to pay the price (I certainly know I've chafed under standards I fought to install).  At that point, things will seem arbitrary and overly constraining, priorities will seem misaligned (and might actually be), and then feelings will be hurt and accusations will be leveled and things will be rough.  The solution there is to have, already in place, strong and open channels of communication, strong norms and scaffolds for emotional support, strong default assumption of trust and good intent on all sides, etc. etc.  This goes wrongest when things fester and people feel they can't speak up; it goes much better if people have channels to lodge their complaints and reservations and are actively incentivized to do so (and can do so without being accused of defecting on the norm-in-question; criticism =/= attack).


Problem 5: Everything else

There are other models and problems in the mix—for instance, I have a model surrounding buy-in and commitment that deals with an escalating cycle of asks-and-rewards, or a model of how to effectively leverage a group around you to accomplish ambitious tasks that requires you to first lay down some "topsoil" of simple/trivial/arbitrary activities that starts the growth of an ecology of affordances, or a theory that the strategy of trying things and doing things outstrips the strategy of think-until-you-identify-worthwhile-action, and that rationalists in particular are crippling themselves through decision paralysis/letting the perfect be the enemy of the good when just doing vaguely interesting projects would ultimately gain them more skill and get them further ahead, or a strong sense based off both research and personal experience that physical proximity matters, and that you can't build the correct kind of strength and flexibility and trust into your relationships without actually spending significant amounts of time with one another in meatspace on a regular basis, regardless of whether that makes tactical sense given your object-level projects and goals.

But I'm going to hold off on going into those in detail until people insist on hearing about them or ask questions/pose hesitations that could be answered by them.

Section 2 of 3: Power dynamics

All of the above was meant to point at reasons why I suspect trusting individuals responding to incentives moment-by-moment to be a weaker and less effective strategy than building an intentional community that Actually Asks Things Of Its Members.  It was also meant to justify, at least indirectly, why a strong guiding hand might be necessary given that our community's evolved norms haven't really produced results (in the group houses) commensurate with the promises of EA and rationality.

Ultimately, though, what matters is not the problems and solutions themselves so much as the light they shine on my aesthetics (since, in the actual house, it's those aesthetics that will be used to resolve epistemic gridlock).  In other words, it's not so much those arguments as it is the fact that Duncan finds those arguments compelling.  It's worth noting that the people most closely involved with this project (i.e. my closest advisors and those most likely to actually sign on as housemates) have been encouraged to spend a significant amount of time explicitly vetting me with regards to questions like "does this guy actually think things through," "is this guy likely to be stupid or meta-stupid," "will this guy listen/react/update/pivot in response to evidence or consensus opposition," and "when this guy has intuitions that he can't explain, do they tend to be validated in the end?"

In other words, it's fair to view this whole post as an attempt to prove general trustworthiness (in both domain expertise and overall sanity), because—well—that's what it is.  In milieu like the military, authority figures expect (and get) obedience irrespective of whether or not they've earned their underlings' trust; rationalists tend to have a much higher bar before they're willing to subordinate their decisionmaking processes, yet still that's something this sort of model requires of its members (at least from time to time, in some domains, in a preliminary "try things with benefit of the doubt" sort of way).  I posit that Dragon Army Barracks works (where "works" means "is good and produces both individual and collective results that outstrip other group houses by at least a factor of three") if and only if its members are willing to hold doubt in reserve and act with full force in spite of reservations—if they're willing to trust me more than they trust their own sense of things (at least in the moment, pending later explanation and recalibration on my part or theirs or both).

And since that's a) the central difference between DA and all the other group houses, which are collections of non-subordinate equals, and b) quite the ask, especially in a rationalist community, it's entirely appropriate that it be given the greatest scrutiny.  Likely participants in the final house spent ~64 consecutive hours in my company a couple of weekends ago, specifically to play around with living under my thumb and see whether it's actually a good place to be; they had all of the concerns one would expect and (I hope) had most of those concerns answered to their satisfaction.  The rest of you will have to make do with grilling me in the comments here.


"Why was Tyler Durden building an army?  To what purpose?  For what greater good? Tyler we trusted."


Power and authority are generally anti-epistemic—for every instance of those-in-power defending themselves against the barbarians at the gates or anti-vaxxers or the rise of Donald Trump, there are a dozen instances of them squashing truth, undermining progress that would make them irrelevant, and aggressively promoting the status quo.

Thus, every attempt by an individual to gather power about themselves is at least suspect, given regular ol' incentive structures and regular ol' fallible humans.  I can (and do) claim to be after a saved world and a bunch of people becoming more the-best-versions-of-themselves-according-to-themselves, but I acknowledge that's exactly the same claim an egomaniac would make, and I acknowledge that the link between "Duncan makes all his housemates wake up together and do pushups" and "the world is incrementally less likely to end in gray goo and agony" is not obvious.

And it doesn't quite solve things to say, "well, this is an optional, consent-based process, and if you don't like it, don't join," because good and moral people have to stop and wonder whether their friends and colleagues with slightly weaker epistemics and slightly less-honed allergies to evil are getting hoodwinked.  In short, if someone's building a coercive trap, it's everyone's problem.


"Over and over he thought of the things he did and said in his first practice with his new army. Why couldn't he talk like he always did in his evening practice group? No authority except excellence. Never had to give orders, just made suggestions. But that wouldn't work, not with an army. His informal practice group didn't have to learn to do things together. They didn't have to develop a group feeling; they never had to learn how to hold together and trust each other in battle. They didn't have to respond instantly to command.

And he could go to the other extreme, too. He could be as lax and incompetent as Rose the Nose, if he wanted. He could make stupid mistakes no matter what he did. He had to have discipline, and that meant demanding—and getting—quick, decisive obedience. He had to have a well-trained army, and that meant drilling the soldiers over and over again, long after they thought they had mastered a technique, until it was so natural to them that they didn't have to think about it anymore."


But on the flip side, we don't have time to waste.  There's existential risk, for one, and even if you don't buy ex-risk à la AI or bioterrorism or global warming, people's available hours are trickling away at the alarming rate of one hour per hour, and none of us are moving fast enough to get All The Things done before we die.  I personally feel that I am operating far below my healthy sustainable maximum capacity, and I'm not alone in that, and something like Dragon Army could help.

So.  Claims, as clearly as I can state them, in answer to the question "why should a bunch of people sacrifice non-trivial amounts of their autonomy to Duncan?"

1. Somebody ought to run this, and no one else will.  On the meta level, this experiment needs to be run—we have like twenty or thirty instances of the laissez-faire model, and none of the high-standards/hardcore one, and also not very many impressive results coming out of our houses.  Due diligence demands investigation of the opposite hypothesis.  On the object level, it seems uncontroversial to me that there are goods waiting on the other side of the unpleasant valley—goods that a team of leveled-up, coordinated individuals with bonds of mutual trust can seize that the rest of us can't even conceive of, at this point, because we don't have a deep grasp of what new affordances appear once you get there.

2. I'm the least unqualified person around.  Those words are chosen deliberately, for this post on "less wrong."  I have a unique combination of expertise that includes being a rationalist, sixth grade teacher, coach, RA/head of a dormitory, ringleader of a pack of hooligans, member of two honor code committees, curriculum director, obsessive sci-fi/fantasy nerd, writer, builder, martial artist, parkour guru, maker, and generalist.  If anybody's intuitions and S1 models are likely to be capable of distinguishing the uncanny valley from the real deal, I posit mine are.

3. There's never been a safer context for this sort of experiment.  It's 2017, we live in the United States, and all of the people involved are rationalists.  We all know about NVC and double crux, we're all going to do Circling, we all know about Gendlin's Focusing, and we've all read the Sequences (or will soon).  If ever there was a time to say "let's all step out onto the slippery slope, I think we can keep our balance," it's now—there's no group of people better equipped to stop this from going sideways.

4. It does actually require a tyrant. As a part of a debrief during the weekend experiment/dry run, we went around the circle and people talked about concerns/dealbreakers/things they don't want to give up.  One interesting thing that popped up is that, according to consensus, it's literally impossible to find a time of day when the whole group could get together to exercise.  This happened even with each individual being willing to make personal sacrifices and doing things that are somewhat costly.

If, of course, the expectation is that everybody shows up on Tuesday and Thursday evenings, and the cost of not doing so is not being present in the house, suddenly the situation becomes simple and workable.  And yes, this means some kids left behind (ctrl+f), but the whole point of this is to be instrumentally exclusive and consensually high-commitment.  You just need someone to make the actual final call—there are too many threads for the coordination problem of a house of this kind to be solved by committee, and too many circumstances in which it's impossible to make a principled, justifiable decision between 492 almost-indistinguishably-good options.  On top of that, there's a need for there to be some kind of consistent, neutral force that sets course, imposes consistency, resolves disputes/breaks deadlock, and absorbs all of the blame for the fact that it's unpleasant to be forced to do things you know you ought to but don't want to do.

And lastly, we (by which I indicate the people most likely to end up participating) want the house to do stuff—to actually take on projects of ambitious scope, things that require ten or more talented people reliably coordinating for months at a time.  That sort of coordination requires a quarterback on the field, even if the strategizing in the locker room is egalitarian.

5. There isn't really a status quo for power to abusively maintain.  Dragon Army Barracks is not an object-level experiment in making the best house; it's a meta-level experiment attempting (through iteration rather than armchair theorizing) to answer the question "how best does one structure a house environment for growth, self-actualization, productivity, and social synergy?"  It's taken as a given that we'll get things wrong on the first and second and third try; the whole point is to shift from one experiment to the next, gradually accumulating proven-useful norms via consensus mechanisms, and the centralized power is mostly there just to keep the transitions smooth and seamless.  More importantly, the fundamental conceit of the model is "Duncan sees a better way, which might take some time to settle into," but after e.g. six months, if the thing is not clearly positive and at least well on its way to being self-sustaining, everyone ought to abandon it anyway.  In short, my tyranny, if net bad, has a natural time limit, because people aren't going to wait around forever for their results.

6. The experiment has protections built in.  Transparency, operationalization, and informed consent are the name of the game; communication and flexibility are how the machine is maintained.  Like the Constitution, Dragon Army's charter and organization are meant to be "living documents" that constrain change only insofar as they impose reasonable limitations on how wantonly change can be enacted.

Section 3 of 3: Dragon Army Charter (DRAFT)

Statement of purpose:

Dragon Army Barracks is a group housing and intentional community project which exists to support its members socially, emotionally, intellectually, and materially as they endeavor to improve themselves, complete worthwhile projects, and develop new and useful culture, in that order.  In addition to the usual housing commitments (i.e. rent, utilities, shared expenses), its members will make limited and specific commitments of time, attention, and effort averaging roughly 90 hours a month (~1.5hr/day plus occasional weekend activities).

Dragon Army Barracks will have an egalitarian, flat power structure, with the exception of a commander (Duncan Sabien) and a first officer (Eli Tyre).  The commander's role is to create structure by which the agreed-upon norms and standards of the group shall be discussed, decided, and enforced, to manage entry to and exit from the group, and to break epistemic gridlock/make decisions when speed or simplification is required.  The first officer's role is to manage and moderate the process of building consensus around the standards of the Army—what they are, and in what priority they should be met, and with what consequences for failure.  Other "management" positions may come into existence in limited domains (e.g. if a project arises, it may have a leader, and that leader will often not be Duncan or Eli), and will have their scope and powers defined at the point of creation/ratification.

Initial areas of exploration:

The particular object level foci of Dragon Army Barracks will change over time as its members experiment and iterate, but at first it will prioritize the following:

  • Physical proximity (exercising together, preparing and eating meals together, sharing a house and common space)
  • Regular activities for bonding and emotional support (Circling, pair debugging, weekly retrospective, tutoring/study hall)
  • Regular activities for growth and development (talk night, tutoring/study hall, bringing in experts, cross-pollination)
  • Intentional culture (experiments around lexicon, communication, conflict resolution, bets & calibration, personal motivation, distribution of resources & responsibilities, food acquisition & preparation, etc.)
  • Projects with "shippable" products (e.g. talks, blog posts, apps, events; some solo, some partner, some small group, some whole group; ranging from short-term to year-long)
  • Regular (every 6-10 weeks) retreats to learn a skill, partake in an adventure or challenge, or simply change perspective

Dragon Army Barracks will begin with a move-in weekend that will include ~10 hours of group bonding, discussion, and norm-setting.  After that, it will enter an eight-week bootcamp phase, in which each member will participate in at least the following:

  • Whole group exercise (90min, 3x/wk, e.g. Tue/Fri/Sun)
  • Whole group dinner and retrospective (120min, 1x/wk, e.g. Tue evening)
  • Small group baseline skill acquisition/study hall/cross-pollination (90min, 1x/wk)
  • Small group circle-shaped discussion (120min, 1x/wk)
  • Pair debugging or rapport building (45min, 2x/wk)
  • One-on-one check-in with commander (20min, 2x/wk)
  • Chore/house responsibilities (90min distributed)
  • Publishable/shippable solo small-scale project work with weekly public update (100min distributed)

... for a total time commitment of 16h/week or 128 hours total, followed by a whole group retreat and reorientation.  The house will then enter an eight-week trial phase, in which each member will participate in at least the following:

  • Whole group exercise (90min, 3x/wk)
  • Whole group dinner, retrospective, and plotting (150min, 1x/wk)
  • Small group circling and/or pair debugging (120min distributed)
  • Publishable/shippable small group medium-scale project work with weekly public update (180min distributed)
  • One-on-one check-in with commander (20min, 1x/wk)
  • Chore/house responsibilities (60min distributed)
... for a total time commitment of 13h/week or 104 hours total, again followed by a whole group retreat and reorientation.  The house will then enter a third phase where commitments will likely change, but will include at a minimum whole group exercise, whole group dinner, and some specific small-group responsibilities, either social/emotional or project/productive (once again ending with a whole group retreat).  At some point between the second and third phase, the house will also ramp up for its first large-scale project, which is yet to be determined but will be roughly on the scale of putting on a CFAR workshop in terms of time and complexity.

Should the experiment prove successful past its first six months, and worth continuing for a full year or longer, by the end of the first year every Dragon shall have a skill set including, but not limited to:
  • Above-average physical capacity
  • Above-average introspection
  • Above-average planning & execution skill
  • Above-average communication/facilitation skill
  • Above-average calibration/debiasing/rationality knowledge
  • Above-average scientific lab skill/ability to theorize and rigorously investigate claims
  • Average problem-solving/debugging skill
  • Average public speaking skill
  • Average leadership/coordination skill
  • Average teaching and tutoring skill
  • Fundamentals of first aid & survival
  • Fundamentals of financial management
  • At least one of: fundamentals of programming, graphic design, writing, A/V/animation, or similar (employable mental skill)
  • At least one of: fundamentals of woodworking, electrical engineering, welding, plumbing, or similar (employable trade skill)
Furthermore, every Dragon should have participated in:
  • At least six personal growth projects involving the development of new skill (or honing of prior skill)
  • At least three partner- or small-group projects that could not have been completed alone
  • At least one large-scale, whole-army project that either a) had a reasonable chance of impacting the world's most important problems, or b) caused significant personal growth and improvement
  • Daily contributions to evolved house culture
Speaking of evolved house culture...

Because of both a) the expected value of social exploration and b) the cumulative positive effects of being in a group that's trying things regularly and taking experiments seriously, Dragon Army will endeavor to adopt no fewer than one new experimental norm per week.  Each new experimental norm should have an intended goal or result, an informal theoretical backing, and a set re-evaluation time (default three weeks).  There are two routes by which a new experimental norm is put into place:

  • The experiment is proposed by a member, discussed in a whole group setting, and meets the minimum bar for adoption (>60% of the Army supports, with <20% opposed and no hard vetos)
  • The Army has proposed no new experiments in the previous week, and the Commander proposes three options.  The group may then choose one by vote/consensus, or generate three new options, from which the Commander may choose.
Examples of some of the early norms which the house is likely to try out from day one (hit the ground running):
  • The use of a specific gesture to greet fellow Dragons (house salute)
  • Various call-and-response patterns surrounding house norms (e.g. "What's rule number one?" "PROTECT YOURSELF!")
  • Practice using hook, line, and sinker in social situations (three items other than your name for introductions)
  • The anti-Singer rule for open calls-for-help (if Dragon A says "hey, can anyone help me with X?" the responsibility falls on the physically closest housemate to either help or say "Not me/can't do it!" at which point the buck passes to the next physically closest person)
  • An "interrupt" call that any Dragon may use to pause an ongoing interaction for fifteen seconds
  • A "culture of abundance" in which food and leftovers within the house are default available to all, with exceptions deliberately kept as rare as possible
  • A "graffiti board" upon which the Army keeps a running informal record of its mood and thoughts

Dragon Army Code of Conduct
While the norms and standards of Dragon Army will be mutable by design, the following (once revised and ratified) will be the immutable code of conduct for the first eight weeks, and is unlikely to change much after that.

  1. A Dragon will protect itself, i.e. will not submit to pressure causing it to do things that are dangerous or unhealthy, nor wait around passively when in need of help or support (note that this may cause a Dragon to leave the experiment!).
  2. A Dragon will take responsibility for its actions, emotional responses, and the consequences thereof, e.g. if late will not blame bad luck/circumstance, if angry or triggered will not blame the other party.
  3. A Dragon will assume good faith in all interactions with other Dragons and with house norms and activities, i.e. will not engage in strawmanning or the horns effect.
  4. A Dragon will be candid and proactive, e.g. will give other Dragons a chance to hear about and interact with negative models once they notice them forming, or will not sit on an emotional or interpersonal problem until it festers into something worse.
  5. A Dragon will be fully present and supportive when interacting with other Dragons in formal/official contexts, i.e. will not engage in silent defection, undermining, halfheartedness, aloofness, subtle sabotage, or other actions which follow the letter of the law while violating the spirit.  Another way to state this is that a Dragon will practice compartmentalization—will be able to simultaneously hold "I'm deeply skeptical about this" alongside "but I'm actually giving it an honest try," and postpone critique/complaint/suggestion until predetermined checkpoints.  Yet another way to state this is that a Dragon will take experiments seriously, including epistemic humility and actually seeing things through to their ends rather than fiddling midway.
  6. A Dragon will take the outside view seriously, maintain epistemic humility, and make subject-object shifts, i.e. will act as a behaviorist and agree to judge and be judged on the basis of actions and revealed preferences rather than intentions, hypotheses, and assumptions (this one's similar to #2 and hard to put into words, but for example, a Dragon who has been having trouble getting to sleep but has never informed the other Dragons that their actions are keeping them awake will agree that their anger and frustration, while valid internally, may not fairly be vented on those other Dragons, who were never given a chance to correct their behavior).  Another way to state this is that a Dragon will embrace the maxim "don't believe everything that you think."
  7. A Dragon will strive for excellence in all things, modified only by a) prioritization and b) doing what is necessary to protect itself/maximize total growth and output on long time scales.
  8. A Dragon will not defect on other Dragons.
There will be various operationalizations of the above commitments into specific norms (e.g. a Dragon will read all messages and emails within 24 hours, and if a full response is not possible within that window, will send a short response indicating when the longer response may be expected) that will occur once the specific members of the Army have been selected and have individually signed on.  Disputes over violations of the code of conduct, or confusions about its operationalization, will first be addressed one-on-one or in informal small group, and will then move to general discussion, and then to the first officer, and then to the commander.

Note that all of the above is deliberately kept somewhat flexible/vague/open-ended/unsettled, because we are trying not to fall prey to GOODHART'S DEMON.

Random Logistics
  1. The initial filter for attendance will include a one-on-one interview with the commander (Duncan), who will be looking for a) credible intention to put forth effort toward the goal of having a positive impact on the world, b) likeliness of a strong fit with the structure of the house and the other participants, and c) reliability à la financial stability and ability to commit fully to long-term endeavors.  Final decisions will be made by the commander and may be informally questioned/appealed but not overruled by another power.
  2. Once a final list of participants is created, all participants will sign a "free state" contract of the form "I agree to move into a house within five miles of downtown Berkeley (for length of time X with financial obligation Y) sometime in the window of July 1st through September 30th, conditional on at least seven other people signing this same agreement."  At that point, the search for a suitable house will begin, possibly with delegation to participants.
  3. Rents in that area tend to run ~$1100 per room, on average, plus utilities, plus a 10% contribution to the general house fund.  Thus, someone hoping for a single should, in the 85th percentile worst case, be prepared to make a ~$1400/month commitment.  Similarly, someone hoping for a double should be prepared for ~$700/month, and someone hoping for a triple should be prepared for ~$500/month, and someone hoping for a quad should be prepared for ~$350/month.
  4. The initial phase of the experiment is a six month commitment, but leases are generally one year.  Any Dragon who leaves during the experiment is responsible for continuing to pay their share of the lease/utilities/house fund, unless and until they have found a replacement person the house considers acceptable, or have found three potential viable replacement candidates and had each one rejected.  After six months, should the experiment dissolve, the house will revert to being simply a house, and people will bear the normal responsibility of "keep paying until you've found your replacement."  (This will likely be easiest to enforce by simply having as many names as possible on the actual lease.)
  5. Of the ~90hr/month, it is assumed that ~30 are whole-group, ~30 are small group or pair work, and ~30 are independent or voluntarily-paired work.  Furthermore, it is assumed that the commander maintains sole authority over ~15 of those hours (i.e. can require that they be spent in a specific way consistent with the aesthetic above, even in the face of skepticism or opposition).
  6. We will have an internal economy whereby people can trade effort for money and money for time and so on and so forth, because heck yeah.

Conclusion: Obviously this is neither complete nor perfect.  What's wrong, what's missing, what do you think?  I'm going to much more strongly weight the opinions of Berkelyans who are likely to participate, but I'm genuinely interested in hearing from everyone, particularly those who notice red flags (the goal is not to do anything stupid or meta-stupid).  Have fun tearing it up.

(sorry for the abrupt cutoff, but this was meant to be published Monday and I've just ... not ... been ... sleeping ... to get it done)

Against lone wolf self-improvement

27 cousin_it 07 July 2017 03:31PM

LW has a problem. Openly or covertly, many posts here promote the idea that a rational person ought to be able to self-improve on their own. Some of it comes from Eliezer's refusal to attend college (and Luke dropping out of his bachelors, etc). Some of it comes from our concept of rationality, that all agents can be approximated as perfect utility maximizers with a bunch of nonessential bugs. Some of it is due to our psychological makeup and introversion. Some of it comes from trying to tackle hard problems that aren't well understood anywhere else. And some of it is just the plain old meme of heroism and forging your own way.

I'm not saying all these things are 100% harmful. But the end result is a mindset of lone wolf self-improvement, which I believe has harmed LWers more than any other part of our belief system.

Any time you force yourself to do X alone in your room, or blame yourself for not doing X, or feel isolated while doing X, or surf the web to feel some human contact instead of doing X, or wonder if X might improve your life but can't bring yourself to start... your problem comes from believing that lone wolf self-improvement is fundamentally the right approach. That belief is comforting in many ways, but noticing it is enough to break the spell. The fault wasn't with the operator all along. Lone wolf self-improvement doesn't work.

Doesn't work compared to what? Joining a class. With a fixed schedule, a group of students, a teacher, and an exam at the end. Compared to any "anti-akrasia technique" ever proposed on LW or adjacent self-help blogs, joining a class works ridiculously well. You don't need constant willpower: just show up on time and you'll be carried along. You don't get lonely: other students are there and you can't help but interact. You don't wonder if you're doing it right: just ask the teacher.

Can't find a class? Find a club, a meetup, a group of people sharing your interest, any environment where social momentum will work in your favor. Even an online community for X that will reward your progress with upvotes is much better than going X completely alone. But any regular meeting you can attend in person, which doesn't depend on your enthusiasm to keep going, is exponentially more powerful.

Avoiding lone wolf self-improvement seems like embarrassingly obvious advice. But somehow I see people trying to learn X alone in their rooms all the time, swimming against the current for years, blaming themselves when their willpower isn't enough. My message to such people: give up. Your brain is right and what you're forcing it to do is wrong. Put down your X, open your laptop, find a class near you, send them a quick email, and spend the rest of the day surfing the web. It will be your most productive day in months.

Bet or update: fixing the will-to-wager assumption

26 cousin_it 07 June 2017 03:03PM

(Warning: completely obvious reasoning that I'm only posting because I haven't seen it spelled out anywhere.)

Some people say, expanding on an idea of de Finetti, that Bayesian rational agents should offer two-sided bets based on their beliefs. For example, if you think a coin is fair, you should be willing to offer anyone a 50/50 bet on heads (or tails) for a penny. Jack called it the "will-to-wager assumption" here and I don't know a better name.

In its simplest form the assumption is false, even for perfectly rational agents in a perfectly simple world. For example, I can give you my favorite fair coin so you can flip it and take a peek at the result. Then, even though I still believe the coin is fair, I'd be a fool to offer both sides of the wager to you, because you'd just take whichever side benefits you (since you've seen the result and I haven't). That objection is not just academic, using your sincere beliefs to bet money against better informed people is a bad idea in real world markets as well.

Then the question arises, how can we fix the assumption so it still says something sensible about rationality? I think the right fix should go something like this. If you flip a coin and peek at the result, then offer me a bet at 90:10 odds that the coin came up heads, I must either accept the bet or update toward believing that the coin indeed came up heads, with at least these odds. I don't get to keep my 50:50 beliefs about the coin and refuse the bet at the same time. More generally, a Bayesian rational agent offered a bet (by another agent who might have more information) must either accept the bet or update their beliefs so the bet becomes unprofitable. The old obligation about offering two-sided bets on all your beliefs is obsolete, use this one from now on. It should also come in handy in living room Bayesian scuffles, throwing some money on the table and saying "bet or update!" has a nice ring to it.

What do you think?

Concrete Ways You Can Help Make the Community Better

22 deluks917 17 June 2017 03:03AM

There is a TLDR at the bottom

Lots of people really value the lesswrong community but aren't sure how to contribute. The rationalist community can be intimidating. We have a lot of very smart people and the standards can be high. Nonetheless there are lots of concrete ways a normal rationalist can help improve the community. I will focus on two areas - engaging with content and a list of shovel ready projects you can get involved in. I will also briefly mention some more speculative ideas at the end of the post.

1) Engaging with Content:

I have spoken to many people I consider great content creators (ex: Zvi, Putanumonit, tristanm). It’s very common to wish their articles got more comments and engagement. The easiest thing you can do is make a lesswrong account and use the upvote button. Seeing upvotes really does motivate good writers. This only works for lesswrong/reddit but it makes a difference. I can think of several lw articles with less upvotes than people who have personally told me the article was great (ex: norm-one-principle by tristanm [1]).

Good comments tend to be even more appreciated than upvotes, and comments can be left on blog posts. If a post has few comments, then almost any decent quality comment is likely to be appreciated by the author. If you have a question or concern, just ask. Many great authors read all their comments, at least those left in the first few days, and often respond to them. Lots of readers comment very rarely, if at all. 95.1% of people who took the SSC survey comment less than once a month and 73.6% never comment at all [2]. The survey showed that survey takers were a highly engaged group who had read lots of posts. If a blog has very few comments I think you should update heavily towards “it’s a good idea for me to post my comment”.

However, what is most lacking in the rational-sphere is positive engagement with non-controversial content you enjoyed.  Recently the SSC sub-reddit found that about 95% of recent content was either in the culture-war thread or contained in a few threads the community considered low quality (based on vote counts) [3]. You can see a similar effect on lesswrong by considering the Dragon Army post [4]. Most good articles posted recently to lesswrong get around 10 comments or less. The Dragon Army post got over 550. I am explicitly not asking people to avoid posting in controversial threads; doing so would be asking a lot of people. But “engagement” is an important reward mechanism for content creators. I do think we should reward more of the writers we find valuable by responding to them with positive engagement.

It’s often difficult to write a comment on a post that you agree with that isn't just “+1 nice post.” Here are some strategies I have found useful:

- If the post is somewhat theoretical try to apply it in a concrete case. Talk about what difficulties you run into and what seems to work well.

- Talk about how the ideas in the post have helped you personally. For example you can say that never understood concept X until you read the post.

- Connect the post to other articles or essays. It’s usually not optimal to just post a link. Either summarize the other article or include a relevant, possibly extended, quote. Reading articles takes time.

- Speculate a little on how the ideas in the article could be extended further.

It’s not just article writers who enjoy people engaging with their work. People who write comments also appreciate getting good responses. Posting high quality comments, including responses to other comments, encourages other people to engage more. You can personally help get a virtuous cycle going. As a side note I am unsure about the relative values of posting a comment directly on a blog vs reposting the blogpost to lesswrong and commenting there. Currently lesswrong is not that inundated with reposts but it could get more crowded in the future. In addition, I think article authors are less likely to read lesswrong comments about their post, but I am not confident in the effect size.

2) Shovel Ready Projects:

-- Set up an online Lesswrong gaming group/server, ideally for a popular game. I have talked to people and Overwatch seems to have a lot of interest. People seemed to think it would really be a blast to play Overwatch with four other rationalists. Another popular idea is Dungeons and Dragons. I am not a gaming expert and lots of games could probably work but I wanted to share the feedback I got. Notably there is already a factorio server [5].

-- Help 'aggregate' a best of rationalist_tumblr effort posts. Rat_Tumblr is very big and hard to follow. Effort posts are mixed in with lots of random things. One could also include the best responses. There is no need to do this on a daily basis. You could just have a blog that only reblogs high-quality effort posts. I would personally follow this blog and would be willing to cooperate in whatever ways I could. I also think this blog would bring some "equality" to rat_Tumb. The structure of tumblr implies that it’s very hard to get readers unless a popular blog interacts with you. People report getting a "year’s worth of activity in a day" when someone like Eliezer or Ozy signal boosts them. An aggregator would be a useful way for less well known blogs to get attention.

-- Help the lesswrong wiki. Currently a decent fraction of lw-wiki posts are fairly out of date. In general the wiki could be doing some exciting thing such as: a distillation of Lesswrong. Fully indexing the diaspora. A list of communities. Spreading rationalist ideas. Rationalist Research. There is currently a project to modernize the wiki [6]. Even if you don't get involved in the more ambitious parts of the wiki you could re-write an article. Re-writing an article doesn't require much commitment and would provide a concrete benefit to the community. The wiki is prominently linked and the community would get a lot of good PR from a polished wiki.

-- Get involved with effective altruism. The Center for Effective Altruism recent posted a very high quality involvement guide [7]. It’s a huge list of concrete actions you can take to get involved. Every action has a brief description and a link to an article. Each article rates the action on time commitment, duration, familiarity and occupation. Very well put together.

-- Get more involved in your local irl rationalist group. Many group leaders (ex: Vanier) have suggested that it can be very hard to get members to lead things. If you are interested in leadership and have a decent reputation your local community might need your help.

I would be very interested in comments suggesting other projects/activities rationalists can get involved with.

3) Conclusion 

As a brief aside I want to mention that I considered writing about outreach. But I don't have tons of experience at outreach and I couldn't really process the data on effective outreach. The subject seems quite complicated. Perhaps someone else has already worked through the evidence. I will however recommend this old article by Paul Christiano (now at open AI) [8]. Notably the camp discussed in this pos did come eventually come into being. It’s not a comprehensive article but it has some good ideas. This guide to “How to Run a Successful Less Wrong Meetup” [9] is extremely polished and has some interesting material related to outreach and attracting new members.

It’s easy to think your actions can't make a difference in the community, but they can. A surprisingly large number of people see comments on lesswrong or r/SSC. Good comments are highly appreciated. The person you befriend and convince to stick around on lesswrong might be the next Scott Alexander. Unfortunately, a lot of the time gratitude and appreciation never gets expressed; I am personally very guilty on this metric. But we are all in this together and this article only covers a small sample of the ways you can help make the community better.

If you have feedback or want any advice/help and don't want to post in public I would be super happy to get your private messages.


- Write more comments on blog posts and non-controversial posts on lw and r/SSC

- Especially consider commenting on posts you agree with

- People are more likely to comment if other people are posting high quality comments.

- Projects: Gaming Server, aggregate tumblr effort-posts, improve lesswrong wiki, leadership local rationalist group

5) References: 





[5] . Modpack:





Bi-Weekly Rational Feed

21 deluks917 24 June 2017 12:07AM

===Highly Recommended Articles:

Introducing The Ea Involvement Guide by The Center for Effective Altruism (EA forum) - A huge list of concrete actions you can take to get involved. Every action has a brief description and a link to an article. Each article rates the action on time commitment, duration, familiarity and occupation. Very well put together.

Deep Reinforcement Learning from Human Preferences - An algorithm learns to backflip with 900 bits of feedback from the human evaluator. "One step towards building safe AI systems is to remove the need for humans to write goal functions, since using a simple proxy for a complex goal, or getting the complex goal a bit wrong, can lead to undesirable and even dangerous behavior. In collaboration with DeepMind’s safety team, we’ve developed an algorithm which can infer what humans want by being told which of two proposed behaviors is better."

Build Baby Build by Bryan Caplan - Quote from a paper estimating the high costs of housing restrictions. We should blame the government, especially local government. The top alternate theory is wrong. Which regulations are doing the damage? It's complicated. Functionalists are wrong. State government is our best hope.

The Use And Abuse Of Witchdoctors For Life by Lou (sam[]zdat) - Anti-bullet magic and collective self-defense. Cultural evolution. People don't directly believe in anti-bullet magic, they believe in elders and witch doctors. Seeing like a State. Individual psychology is the foundation. Many psychologically important customs couldn't adapt to the marketplace.

S-risks: Why They Are The Worst Existential Risks by Kaj Sojata (lesswrong) - “S-risk – One where an adverse outcome would bring about severe suffering on a cosmic scale, vastly exceeding all suffering that has existed on Earth so far.” Why we should focus on S-risk. Probability: Artificial sentience, Lack of communication, badly aligned Ai and competitive pressures. Tractability: Relationship with x-risk. Going meta, cooperation. Neglectedness: little attention, people conflate x-risk = s-risk.

Projects Id Like To See by William MacAskill (EA forum) - CEA is giving out £100K grants. General types of applications. EA outreach and Community, Anti-Debates, Prediction Tournaments, Shark Tank Discussions, Research Groups, Specific Skill Building, New Organizations, Writing.

The Battle For Psychology by Jacob Falkovich (Put A Number On It!) - An explanation of 'power' in statistics and why its always good. Low power means that positive results are mostly due to chance. Extremely bad incentives and research practices in psychology. Studying imaginary effects. Several good images.

Identifying Sources Of Cost Disease by Kurt Spindler - Where is the money going: Administration, Increased Utilization, Decreased Risk Tolerance. What market failures are in effect: Unbounded Domains, Signaling and Competitive Pressure (ex: military spending), R&D doesn't cut costs it creates new ways to spend money, individuals don't pay. Some practical strategies to reduce cost disease.


To Understand Polarization Understand The Extent Of Republican Failure by Scott Alexander - Conservative voters voted for “smaller government”, “fewer regulations”, and “less welfare state”. Their reps control most branches of the government. They got more of all three (probably thanks to cost disease).

Against Murderism by Scott Alexander - Three definitions of racism. Why 'Racism as motivation' fits best. The futility of blaming the murder rate in the USA on 'murderism'. Why its often best to focus on motivations other than racism.

Open Thread Comment by John Nerst (SSC) - Bi-weekly public open thread. I am linking to a very interesting comment. The author made a list of the most statistically over-represented words in the SSC comment section.

Some Unsong Guys by Scott Alexander (Scratchpad) - Pictures of Unsong Fan Art.

Silinks Is Golden by Scott Alexander - Standard SSC links post.

What is Depression Anyway: The Synapse Hypothesis - Six seemingly distinct treatments for depression. How at least six can be explained by considering synapse generation rates. Skepticism that this method can be used to explain anything since the body is so inter-connected. Six points that confuse Scott and deserve more research. Very technical.


Idea For Lesswrong Video Tutoring by adamzerner (lesswrong) - Community Video Tutoring. Sign up to either give or receive tutoring. Teaching others is a good way to learn and lots of people enjoy teaching. Hopefully enough people want to learn similar things. This could be a great community project and I recommend taking a look.

Regulatory Arbitrage For Medical Research What I Know So Far by Sarah Constantin (Otium) - Economics of avoiding the USA/FDA. Lots of research is already conducted in other countries. The USA is too large of a market not to sell to. Investors aren't interested in cheap preliminary trials. Other options: supplements, medical tourism, clinic ships, cryptocurrency.

Responses To Folk Ontologies by Ferocious Truth - Folk ontology: Concepts and categories held by ordinary people with regard to an idea. Especially pre-scientific or unreflective ones. Responses: Transform/Rescue, Deny or Restrict/Recognize. Rescuing free will and failing to rescue personal identity. Rejecting objective morality. Restricting personal identity and moral language. When to use each approach.

The Battle For Psychology by Jacob Falkovich (Put A Number On It!) - An explanation of 'power' in statistics and why its always good. Low power means that positive results are mostly due to chance. Extremely bad incentives and research practices in psychology. Studying imaginary effects. Several good images.

A Tangled Task Future by Robin Hanson - We need to untangle the economy to automate it. What tasks are heavily tangled and which are not. Ems and the human brain as a legacy system. Human brains are well-integrated and good at tangled tasks.

Epistemic Spot Check Update by Aceso Under Glass - Reviewing self-help books. Properties of a good self-help model: As simple as possible but not more so, explained well, testable on a reasonable timescale, seriously handles the fact the techniques might now work, useful. The author would appreciate feedback.

Skin In The Game by Elo (BearLamp) - Armchair activism and philosophy. Questions to ask yourself about your life. Actually do the five minute exercise at the end.

Momentum Reflectiveness Peace by Sarah Constantin (Otium) - Rationality requires a reflective mindset; a willingness to change course and consider how things could be very different. Momentum, keeping things as they are except more so, is the opposite of reflectivity. Cultivating reflectiveness: rest, contentment, considering ideas lightly and abstractly. “Turn — slowly.”

The Fallacy Fork Why Its Time To Get Rid Of by theFriendlyDoomer (r/SSC) - "The main thesis of our paper is that each and every fallacy in the traditional list runs afoul of the Fallacy Fork. Either you construe the fallacy in a clear-cut and deductive fashion, which means that your definition has normative bite, but also that you hardly find any instances in real life; or you relax your formal definition, making it defeasible and adding contextual qualifications, but then your definition loses its teeth. Your “fallacy” is no longer a fallacy."

Instrumental Rationality 1 Starting Advice by lifelonglerner (lesswrong) - "This is the first post in the Instrumental Rationality Sequence. This is a collection of four concepts that I think are central to instrumental rationality-caring about the obvious, looking for practical things, practicing in pieces, and realistic expectations."

Concrete Ways You Can Help Make The Community Better by deluks917 (lesswrong) - Write more comments on blog posts and non-controversial posts on lw and r/SSC. Especially consider commenting on posts you agree with. People are more likely to comment if other people are posting high quality comments. Projects: Gaming Server, aggregate tumblr effort-posts, improve lesswrong wiki, leadership in local rationalist group

Daring Greatly by Bayesian Investor - Fairly positive book review, some chapters were valuable and it was an easy read. How to overcome shame and how it differs from guilt. Perfectionism vs healthy striving. If you stop caring about what others think you lose your capacity for connection

A Call To Adventure by Robin Hanson - Meaning in life can be found by joining or starting a grand project. Two possible adventures: Promoting and implementing futarchy (decision making via prediction markets). Getting a real understanding of human motivation.

Thought Experiment Coarsegrained Vr Utopia by cousin_it (lesswrong) - Assume an AI is running a Vr simulation that is hooked up to actual human brains. This means that the AI only has to simulate nature at a coarse grained level. How hard would it be to make that virtual reality a utopia?

[The Rationalist-sphere and the Lesswrong Wiki]]( - What's next for the Lesswrong wiki. A distillation of Lesswrong. Fully indexing the diaspora. A list of communities. Spreading rationalist ideas. Rationalist Research.

Deep Reinforcement Learning from Human Preferences - An algorithm learns to backflip with 900 bits of feedback from the human evaluator. "One step towards building safe AI systems is to remove the need for humans to write goal functions, since using a simple proxy for a complex goal, or getting the complex goal a bit wrong, can lead to undesirable and even dangerous behavior. In collaboration with DeepMind’s safety team, we’ve developed an algorithm which can infer what humans want by being told which of two proposed behaviors is better."

Where Do Hypotheses Come From by c0rw1n (lesswrong) - Link to a 25 page article. "Why are human inferences sometimes remarkably close to the Bayesian ideal and other times systematically biased? In particular, why do humans make near-rational inferences in some natural domains where the candidate hypotheses are explicitly available, whereas tasks in similar domains requiring the self-generation of hypotheses produce systematic deviations from rational inference. We propose that these deviations arise from algorithmic processes approximating Bayes’ rule."

The Precept Of Universalism by H i v e w i r e d - "Universality, the idea that all humans experience life in roughly the same way. Do not put things or ideas above people. Honor and protect all peoples." Eight points expanding on how to put people first and honor everyone.

We Are The Athenians Not The Spartans by wubbles (lesswrong) - "Our values should be Athenian: individualistic, open, trusting, enamored of beauty. When we build social technology, it should not aim to cultivate values that stand against these. High trust, open, societies are the societies where human lives are most improved."


Updating My Risk Estimate of Geomagnetic Big One by Open Philosophy - Risk from magnetic storms caused by the sun. "I have raised my best estimate of the chance of a really big storm, like the storied one of 1859, from 0.33% to 0.70% per decade. And I have expanded my 95% confidence interval for this estimate from 0.0–4.0% to 0.0–11.6% per decade."

Links by GiveDirectly - Eight Media articles on Cash Transfers, Basic Income and Effective Altruism.

Are Givewells Top Charities The Best Option For Every Donor by The GiveWell Blog - Why GiveWell recommend charities are a good option for most donors. Which donors have better options: Donors with lots of time, high trust in a particular institution or values different from GiveWell's.

A New President of GWWC by Giving What We Can - Julia Wise is the New president of Giving What We Can.

Angst Ennui And Guilt In Effective Altruism by Gordon (Map and Territory) - Learning about existential risk can cause psychological harm. Guilt about being unable to help solve X-risk. Akrasia. Reasons to not be guilty: comparative advantage, ability is unequally distributed.

S-risks: Why They Are The Worst Existential Risks by Kaj Sojata (lesswrong) - “S-risk – One where an adverse outcome would bring about severe suffering on a cosmic scale, vastly exceeding all suffering that has existed on Earth so far.” Why we should focus on S-risk. Probability: Artificial sentience, Lack of communication, badly aligned Ai and competitive pressures. Tractability: Relationship with x-risk. Going meta, cooperation. Neglectedness: little attention, people conflate x-risk = s-risk.

Update On Sepsis Donations Probably Unnecessary by Sarah Constantin (Otium) - Sarah C had asked people to crowdfund a sepsis RCT. The trial will probably get funded by charitable foundations. Diminishing returns. Finding good giving opportunities is hard and talking to people in the know is a good way to find things out.

What Is Valuable About Effective Altruism by Owen_Cotton-Barratt (EA forum) - Why should people join EA? The impersonal and personal perspectives. Tensions and synergies between the two perspectives. Bullet point conclusions for researchers, community leaders and normal members.

QALYs/$ Are More Intuitive Than $/QALYs by ThomasSittler (EA forum) - QALYs/$ are preferable to $/QALYs. visual representations on graphs. Avoiding Small numbers and re-normalizing to QUALs/10K$.

Introducing The Ea Involvement Guide by The Center for Effective Altruism (EA forum) - A huge list of concrete actions you can take to get involved. Every action has a brief description and a link to an article. Each article rates the action on time commitment, duration, familiarity and occupation. Very well put together.

Cash is King by GiveDirectly - Eight media articles about Effective Altruism and Cash transfers.

Separating GiveWell and the Open Philanthropy Project by The GiveWell Blog - The GiveWell perspective. Context for the sale. Effect on donors who rely on GiveWell. Organization changes at GiveWell. Steps taken to sell Open Phil assets. The new relationship between GiveWell and Open Phil.

Open Philanthropy Project is Now an Independent Organization by Open Philosophy - The evolution of Open Phil. Why should Open Phil split from GiveWell. LLC structure.

Projects Id Like To See by William MacAskill (EA forum) - CEA is giving out £100K grants. General types of applications. EA outreach and Community, Anti-Debates, Prediction Tournaments, Shark Tank Discussions, Research Groups, Specific Skill Building, New Organizations, Writing.

===Politics and Economics:

No Us School Funding Is Actually Somewhat Progressive by Random Critical Analysis - Many people think that wealthy public school districts spend more per pupil. This information is outdated. Within most states spending is higher on disadvantaged students. This is despite the fact that school funding is mostly local. Extremely thorough with loads of graphs.

Build Baby Build by Bryan Caplan - Quote from a paper estimating the high costs of housing restrictions. We should blame the government, especially local government. The top alternate theory is wrong. Which regulations are doing the damage? It's complicated. Functionalists are wrong. State government is our best hope.

Identifying Sources Of Cost Disease by Kurt Spindler - Where is the money going: Administration, Increased Utilization, Decreased Risk Tolerance. What market failures are in effect: Unbounded Domains, Signaling and Competitive Pressure (ex: military spending), R&D doesn't cut costs it creates new ways to spend money, individuals don't pay. Some practical strategies to reduce cost disease.

The Use And Abuse Of Witchdoctors For Life by Lou (sam[]zdat) - Anti-bullet magic and collective self-defense. Cultural evolution. People don't directly believe in anti-bullet magic, they believe in elders and witch doctors. Seeing like a State. Individual psychology is the foundation. Many psychologically important customs couldn't adapt to the marketplace.

Greece Gdp Forecasting by João Eira (Lettuce be Cereal) - Transforming the Data. Evaluating the Model with Exponential Smoothing, Bagged ETS and ARIMA. The regression results and forecast.

Links 9 by Artir (Nintil) - Economics, Psychology, Artificial Intelligence, Philosophy and other links.

Amazon Buying Whole Foods by Tyler Cowen - Quotes from Matt Yglesias, Alex Tabarrock, Ross Douthat and Tyler. “Dow opens down 10 points. Amazon jumps 3% after deal to buy Whole Foods. Walmart slumps 7%, Kroger plunges 16%”

Historical Returns Market Portfolio by Tyler Cowen - From 1960 to 2015 the global market portfolio realized a compounded real return of 4.38% with a std of 11.6%. Investors beat savers by 3.24%. Link to the original paper.

Trust And Diver by Bryan Caplan - Robert Putnam's work is often cited as showing the costs of diversity. However Putnam's work shows the negative effect of diversity on trust is rather modest. On the other hand Putnam found multiple variables that are much more correlated with trust (such as home ownership).

Why Optimism is More Rational than Pessimism by TheMoneyIllusion - Splitting 1900-2017 into Good and Bad periods. We learn something from our mistakes. Huge areas where things have improved long term. Top 25 movies of the 21st Century. Artforms in decline.

Is Economics Science by Noah Smith - No one knows what a Science is. Thoeries that work (4 examples). The empirical and credibility revolutions. Why we still need structural models. Ways economics could be more scientific. Data needs to kill bad theories. Slides from Noah's talk are included and worth playing but assume familiarity with the economics profession.


Clojure Concurrency And Blocking With Coreasync by Eli Bendersky - Concurrent applications and blocking operations using core.async. Most of the article compares threads and go-blocks. Lots of code and well presented test results.

Optopt by Ben Kuhn - Startup options are surprisingly valuable once you factor in that you can quit of the startup does badly. A mathematical model of the value of startup options and the optimal time to quit. The ability to quit rose the option value by over 50%. The sensitivity of the analysis with respect to parameters (opportunity cost, volatility, etc).

Epistemic Spot Check: The Demon Under The Microscope by Aceso Under Glass - Biography of the man who invented sulfa drugs, the early anti-bacteria treatments which were replaced by penicillin. Interesting fact checks of various claims.

Sequential Conversion Rates by Chris Stucchio - Estimating success rates when you have noisy reporting. The article is a sketch of how the author handled such a problem in practice.

Set Theory Problem by protokol2020 - Bring down ZFC. Aleph-zero spheres and Aleph-one circles.

Connectome Specific Harmonic Waves On Lsd by Qualia Computing - Transcript and video of a talk on neuroimaging the brain on LSD. "Today thanks to the recent developments in structural neuroimaging techniques such as diffusion tensor imaging, we can trace the long-distance white matter connections in the brain. These long-distance white matter fibers (as you see in the image) connect distant parts of the brain, distant parts of the cortex."

Approval Maximizing Representations by Paul Christiano - Representing images. Manipulation representations. Iterative and compound encodings. Compressed representations. Putting it all together and bootstrapping reinforcement learning.

Travel by Ben Kuhn - Advice for traveling frequently. Sleeping on the plane and taking redeyes. Be robust. Bring extra clothes, medicine, backup chargers and things to read when delayed. Minimize stress. Buy good luggage and travel bags.

Learning To Cooperate, Compete And Communicate by Open Ai - Competitive multi-agent models are a step towards AGI. An algorithm for centralized learning and decentralized execution in multi-agent environment. Initial Research. Next Steps. Lots of visuals demonstrating the algorithm in practice.

Openai Baselines Dqn by Open Ai - "We’re open-sourcing OpenAI Baselines, our internal effort to reproduce reinforcement learning algorithms with performance on par with published results." Best practices we use for correct RL algorithm implementations. First release: DQN and three of its variants, algorithms developed by DeepMind.

Corrigibility by Paul Christiano - Paul defines the sort of AI he wants to build, he refers to such systems as "corrigible". Paul argues that a sufficiently corrigible agent will become more corrigible over time. This implies that friendly AI is not a narrow target but a broad basin of attraction. Corrigible agents prefer to build other agents that share the overseers preferences, not their own. Predicting that the overseer wants me to turn off when he hits the off-button is not complicated relative to being deceitful. Comparison with Eliezer's views.

G Reliant Skills Seem Most Susceptible To Automation by Freddie deBoer - Computers already outperform humans in g-loaded domains such as Go and Chess. Many g-loaded jobs might get automated. Jobs involving soft or people skills are resilient to automation.

Persona 5: Spoiler Free Review - Persona games are long but deeply worthwhile if you enjoy the gameplay and the story. Persona 5 is much more polished but Persona 3 has a more meaningful story and more interesting decisions. Tips for Maximum Enjoyment of Persona 5. Very few spoilers.

Sea Problem by protokol2020 - A fun problem. Measuring sea level rise.


83 The Politics Of Emergency by Waking Up with Sam Harris - Fareed Zakaria. "His career as a journalist, Samuel Huntington's "clash of civilizations," political partisanship, Trump, the health of the news media, the connection between Islam and intolerance"

On Risk, Statistics, And Improving The Public Understanding Of Science by 80,000 Hours - A lifetime of communicating science. Early career advice. Getting people to intuitively understand hazards and their effect on life expectancy.

Ed Luce by Tyler Cowen - The Retreat of Western Liberalism "What a future liberalism will look like, to what extent current populism is an Anglo-American phenomenon, Modi’s India, whether Kubrick, Hitchcock, and John Lennon are overrated or underrated, and what it is like to be a speechwriter for Larry Summers."

Thomas Ricks by EconTalk - Thomas Ricks book Churchill and Orwell. Overlapping lives and the fight to preserve individual liberty.

The End Of The World According To Isis by Waking Up with Sam Harris - Graeme Wood. His experience reporting on ISIS, the myth of online recruitment, the theology of ISIS, the quality of their propaganda, the most important American recruit to the organization, the roles of Jesus and the Anti-Christ in Islamic prophecy, free speech and the ongoing threat of jihadism.

Jason Khalipa by Tim Ferriss - "8-time CrossFit Games competitor, a 3-time Team USA CrossFit member, and — among other athletic feats — he has deadlifted 550 pounds, squatted 450 pounds, and performed 64 pullups at a bodyweight of 210 pounds."

Dario Amodei, Paul Christiano & Alex Ray. - 80K hours released a detailed guide to careers in AI policy. " We discuss the main career paths; what to study; where to apply; how to get started; what topics are most in need of research; and what progress has been made in the field so far." Transcript included.

Don Bourdreaux Emergent Order by EconTalk - "Why is it that people in large cities like Paris or New York City people sleep peacefully, unworried about whether there will be enough bread or other necessities available for purchase the next morning? No one is in charge--no bread czar. No flour czar."

Tania Lombrozo On Why We Evolved The Urge To Explain by Rational Speaking - "Research on what purpose explanation serves -- i.e., why it helps us more than our brains just running prediction algorithms. Tania and Julia also discuss whether simple explanations are more likely to be true, and why we're drawn to teleological explanations"

Bi-weekly Rational Feed

19 deluks917 08 August 2017 01:56PM

===Highly Recommended Articles:

Skills Most Employable by 80,000 Hours - Metrics: Satisfaction, risk of automation, and breadth of applicability. Leadership and social skills will gain the most in value. The least valuable skills involve manual labor. Tech skills may not be the most employable but they are straightforward to improve at. The most valuable skills are the hardest to automate and useful in the most situations. Data showing a large oversupply of some tech skills, though others are in high demand. A chart of which college majors add the most income.

Something Was Wrong by Zvi Moshowitz - Zvi visits a 'stepford pre-school'. He can't shake the feeling that something is wrong. He decides not to send his son to the place where kid's souls go to die.

Ems Evolve by Bayesian Investor - Will the future we dominated by entities that lack properties we consider important (such as 'have fun' or even 'sentient'). Will agents lacking X-value outcompete other agents. What counter-measures could society take and how effective would they be.

Housing Price Bubble Revisited by Tyler Cowen - "Over the entire 20th century real home prices averaged an index value of about 110 (and were quite close to this value over the the entire 1950-1997 period). Over the entire 20th century, housing prices never once roce above 131, the 1989 peak. But beginning around 2000 house prices seemed to reach for an entirely new equilibrium. In fact, even given the financial crisis, prices since 2000 fell below the 20th century peak for only a few months in late 2011. Real prices today are now back to 2004 levels and rising. As I predicted in 2008, prices never returned to their long-run 20th century levels."

Tyler Cowen On Stubborn Attachments by EconTalk - "Cowen argues that economic growth--properly defined--is the moral key to maintaining civilization and promoting human well-being. Along the way, the conversation also deals with inequality, environmental issues, and education"


Contra Grant On Exaggerated Differences by Scott Alexander - "Hyde found moderate or large gender differences in aggressiveness, horniness, language abilities, mechanical abilities, visuospatial skills, mechanical ability, tendermindness, assertiveness, comfort with body, various physical abilities, and computer skills. Perhaps some peeople might think that finding moderate-to-large-differences in mechanical abilities, computer skills, etc supports the idea that gender differences might play a role in gender balance in the tech industry. But because Hyde’s meta-analysis drowns all of this out with stuff about smiling-when-not-observed, Grant is able to make it sound like this proves his point. It’s actually worse than this, because Grant misreports the study findings in various ways."

Links: On The Site Of The Angels by Scott Alexander - Standard SSC links post.

Mildly Condescending Advice by SlateStarScratchpad - Ten mildly condescending but useful pieces of advice Scott recommends.

Communism by SlateStarScratchpad - Scott thinks he would have been a communist in 1910.

What Are The Median Psychiatrists Scores On The by SlateStarScratchpad - Psychiatrists are very mentally well adjusted on average. "I think you get way more illness in the therapists, counselors, etc, especially the ones that are kind of low-status and don’t require a lot of training." Doctor's recovery rates from alcoholism are very good.

Why Not More Excitement About Prediction Aggregation by Scott Alexander - Prediction markets and aggregation methods work. Superforecasters proved some groups can consistently make good predictions. Why isn't there more interest? Wouldn't investors pay for predictions? Do theories about signaling and prestige explain the situation?

Where The Falling Einstein Meets The Rising Mouse by Scott Alexander - Eliezer/Scott's model of intelligence suggests that the gap between 'village idiot' and Einstein is tiny relative to the difference between 'village idiot' and a chimp. This suggests that once AI reaches human levels it will almost immediately pass the best human. This happened in Go. But in other fields progress was gradual throughout the approximately human level skill range. Scott looks at possible explanations.

Stem vs The Humanities by SlateStarScratchpad - A long and intelligent thread about "STEM" vs "The Humanities". What are the natural categories? Should we consider math part of the humanities? Should we groups careful humanities scholars with careful STEM scholars? So-called-autistics. Other topics.

Gender Imbalances Are Mostly Not Due To Offensive Attitudes by Scott Alexander - Men and women massively differ in terms of interest in things vs people. Libertarians are about 5% women. r/MRAand the gamergate subreddit have twice this percentage. Trump voters are close to gender parity and the Catholic Church has more women than men. Why this matters.

Is It Possible To Have Coherent Principles Around Free Speech Norms by Scott Alexander - The doctrine of the preferred first speaker. Separating having an opinion, signaling a propensity, and committing a speech act. Self selected communities. "Don’t try to destroy people in order to enforce social norms that only exist in your head"

Book Review Raise A Genius by Scott Alexander - Scott quote-mines Polgar's book on raising genius. Many of the quotes are concerned with the importance of instilling a love of learning in children. Polgar gives some detail on how to do this but not as much as Scott hoped. Summary: "Get those four things right – early start, single-subject focus, 1:1 home schooling, and a great parent/teacher – and the rest is just common-sense advice."

Against Signal Boosting As Doxxing by Scott Alexander - Free speech did not come into existence ex nihilo when the First Amendment was ratified. People need to be free from mobs as well as kings.

Open Djed by Scott Alexander - Bi-Weekly public open thread. Meetup Tab. Updates on rationalist houses in Berkeley. Selected comments on Griggs. A comment on Democratic strategy and Georgia.

Why Is Clozapine So Great by Scott Alexander - Clozapine is a very effect anti-psychotic but it has large serious side effects. NDMA agonists and a proposed mechanism for Clozapine. Maybe you could just give patients a normal anti-psychotic plus glycine.

Djoser Joseph Osiris by Scott Alexander - "The other day a few people (including Ben Hoffman of Compass Rose) tried to convince me that Pharaoh Djoser was the inspiration for the god Osiris and the Biblical Joseph. The short summary is that the connection between Djoser and Osiris is probably meaningless, but there’s a very small chance there might be some tiny distant scrap of a connection to Joseph."

Don't Blame Griggs by Scott Alexander - Griggs vs. Duke Power Co is commonly cited as making it prohibitively hard for companies to use intelligence tests in hiring. Scott argues this doesn't explain the rise in credentialism. You can still ask about SAT scores. Fields with easily available test scores (LSAT, MCAT) are still credentialist. Other countries lack equivalents of Griggs vs Duke.

Highlights From The Comment Thread On Meritocracy by Scott Alexander - Real merit vs credentials. Which merits do we reward? Meritocracy causes high ability people to concentrate into one class. Just rid rid of ruler and structural divisions between people. Scott finds the later idea utopian. " The most salient alternative to meritocracy isn’t perfect equality, it’s cronyism."


Why So Few Women In Cs: The Google Memo Is Right by Artir (Nintil) - Sampler: Lots of data and graphs covering multiple countries. "Occupational segregation by gender is stronger in egalitarian countries. This is a fatal blow to the sexism theory." In the 1980s demand for the CS major far outstripped capacity. This lead to severe limits on who could major in CS. These limits occurred at the same time female enrollment percentage dropped.

Double Crux Web App by mindlevelup - Double Crux is a rationalist technique for resolving and understanding disagreements. It involves identifying facts/statements, called cruxes, that would cause you to change your mind if you changed your mind about the crux. The author built software to facilitate double crux during the Google CSSI 3 week web dev camp. Links to the site and an explanation of Double Crux.

Compare Institutions To Institutions Not To Perfection by Robin Hanson - Hanson responds to criticisms of prediction markets. Short term accuracy is always easier to incentivize. Its always easier to find surface as opposed to deep connections.

Thank You For Listening by Ben Hoffman (Compass Rose) - Zvi's post above starts with a reference to a previous Ben Quo post. If you have hurt your child via school you aren't the enemy. Society taught you that you were helping. If you are still sending your child to a harmful system you aren't the enemy either, you are doing what you think is best.

Something Was Wrong by Zvi Moshowitz - Zvi visits a 'stepford pre-school'. He can't shake the feeling that something is wrong. He decides not to send his son to the place where kid's souls go to die.

Inscrutable Ideas by Gordon (Map and Territory) - The author describes 'holonic' thinking and why its hard to explain. Postmodernism as a flawed holonic tradition. Buddhism as a better holonic tradition. Fundamental incompatibility with system-relationship epistemology.

Body Pleasure by Sarah Perry (ribbonfarm) - "As non-human intelligences get more sophisticated, it may be the case that human work remains extremely important; however, it may also be that humans are faced with increasing leisure. If that is the case, the critical problem facing humanity will be how to enjoy ourselves. If that seems silly, consider your favorite dystopian images of the future: only humans who understand how to enjoy themselves can demand living conditions in which they are able to do so."

Erisology Of Self And Will The Need And The Reasons by Everything Studies - "Here in part 6 I discuss the reasons why the traditional view persists when prescientific thinking on other topics often doesn’t."

Confidence And Patience Dont Feel Like Anything In Particular by Kaj Sotala - Being confident doesn't feel like anything. 'Feeling confident' is really just the lack of feeling unconfident.

Foom Justifies Ai Risk Efforts Now by Robin Hanson - Organizations and corporations are already much smarter and more powerful than individuals, yet they remain mostly under control. Despite setbacks (Wars, revolutions, famines) the organization ecosystem is mostly functional. The only reason to be preemptively worried about AI is if AI takeoff will be very fast.

Skills Most Employable by 80,000 Hours - Metrics: Satisfaction, risk of automation, and breadth of applicability. Leadership and social skills will gain the most in value. The least valuable skills involve manual labor. Tech skills may not be the most employable but they are straightforward to improve at. The most valuable skills are the hardest to automate and useful in the most situations. Data showing a large oversupply of some tech skills, though others are in high demand. A chart of which college majors add the most income.

A Tactics by protokol2020 - Why its very hard to argue against the scientific consensus on fields such as Relativity or Quantum Mechanics. The Earth's temperature was hotter when it rotated fast, despite a fainter sun. Many physicists failed to grasp this fact. What does that imply?

Hedonic Model by Jeff Kaufman - "Happiness is having how things are going be closer to how you think things could be going." Some interesting implications including that both inequality and social mobility are bad.

Link Blog: Broadcom Broadpwn Gender Signal by Name and Nature - Links: History of Atheism. Evolution of Trust. Graphics depicting the Fast Fourier Transform. Remotely Compromising Android and iOS via a Bug in Broadcom’s Wi-Fi Chipsets.

Lying On The Ground by mindlevelup - "A rambling look at how rewards, distractions, and attention interact. Starts with the idea of lying on the ground as an interesting break-time activity and goes from there to talk about Saturation and Feeling, two concepts that I’ve been thinking about lately."

Ems Evolve by Bayesian Investor - Will the future we dominated by entities that lack properties we consider important (such as 'have fun' or even 'sentient'). Will agents lacking X-value outcompete other agents. What counter-measures could society take and how effective would they be.

Models Of Human Relationships Tools To Understand People by Elo (BearLamp) - Brief Model descriptions: Crucial Conversations. 4 Difficult Conversations, 4 Behaviors that kill relationships, How to Win Friends and Influence People (Detailed review), Non-Judgmental conversations, Emotional Intelligence. Circling, The Game (PUA), Apologies, Emotional Labor and others.

Inefficiencies In The Social Value Market by Julia Galef - Add liquidity where needed. Solve coordination problems. Pool risks. Provide resource allocation information. Make biases work for you. Remove rent-seeking. Reduce transaction costs.

Erisology Of Self And Will Campbellian Thinking In The Wild by Everything Studies - "In this section I’ll show some examples of casual conversation revealing Campbellian ideas. Comment threads attached to online newspaper articles are excellent sources of such casual conversation. Written down in a neat and accessible form, their existence makes it practical to do this kind of research for the first time."

The Future: Near Zero Growth Rates by The Foundational Research Institute - Moore's law cannot possibly go one for more than ~400 years, we will hit physical limits to computation. At 2.3% growth in energy use we would need to coat the Earth in Solar panels to get enough solar energy in only 400 years. If we captured all the energy from the sun we would run out in 1350 years. The universe can only support so much economic activity. We will in a very unusual part of humanity's timeline in terms of growth rates.

How I Found Fixed The Root Problem Behind My Depression And Anxiety After 20 Years by Kaj Sotala- Finding the root cause: self-concept. How to cultivate lovable self-concepts (ex: bravery). Consider memories where you lived up to the concept of being brave. Also consider cases where you failed. Integrating the positives and negatives into a healthy whole. Positive benefits the author experienced: professional success, emotional landscape improvement, negative emotions disappeared. Expected relationship changes. Lots of personal history details throughout.

Taking Integrity Literally by Ben Hoffman (Compass Rose) - Defending Kant. Fight the murderer or shut the door but don't become the sort of person who considers lying. Honesty is optimal in healthy environments. Thoughts on unhealthy environments. How Ben started to become honest about how late he would be. Not lying to yourself or others.

People Dont Have Beliefs Anymore Than They Have by Bound_up (lesswrong) - Actions are not deduced form goals. Beliefs are not deduced from models of the world. Maybe nerds have real beliefs but most people do not. Less nerdy people will probably interpret in your stated beliefs as social moves and will respond in turn.

Complexity Is Bad by Zvi Moshowitz - People can only think about ~3 things and store ~7 pieces of information in working memory. People will simplify in unexpected ways or fail to engage. Some concepts that help you manage complexity (ex: Resonance, Chunking). A link to the MtG head of R&D's podcast about why complexity is a cost.

Write Down Your Process by Zvi Moshowitz - Writing down your thought process helps you improve. Magic R&D's openness. Zvi's success as a MtG player and writer.


July 2017 Newsletter by The MIRI Blog - News and Links: Open AI, Deepmind, AI Impacts, EA global, 80K hours, etc

Yudkowsky And Miri by Jeff Kaufman - Elizier once wrote an extremely embarrassing article called 'So you want to be a Seed AI Programmer'. A ML researcher showed it to Jeff Kaufman and said it implied Elizier was a crank. Elizier wrote it in 2003, when he was 24. What does this imply about MIRI?


Medical Research Cancer Is Hugely Overfunded by Sanjay (EA forum) - Chart of disease burden vs research share. Six reasons you might disagree with the conclusion including cause tractability and methodology.

Blood Donation Generally Not That Effective On by Grue_Slinky (EA forum) - Having a supply of blood is very important. However the marginal value of blood donation is too low to recommend it as an efficient intervention.

How We Banned Fur In Berkeley by jayquigly (EA forum) - Fur sales banned. Main strategies: cultivating relationships with sympathetic council members, utilizing a proven template for the bill. Background. Strategy Details. Advice

Links: Our Main Goal is to Learn by GiveDirectly - Eight media links on Give Directly, Basic Income, Cash Transfers and Development Aid.

Funding Constraints For Ea Orgs by Jeff Kaufman - Value of direct work vs donation. Jeff argues EA organizations could make use of more resources. For example EA-Global could hire non-EA professional conference organizers.

===Politics and Economics:

Rise And Fall Of Rawlsianism by Artir (Nintil) - "I will introduce street Rawlsianism, a simplified version of Rawls’s Theory of Justice to get an idea of what this is all about. Then, I will explain how that came to be, including some extra details about Rawls’s justification for his theory. This story itself, the development of Rawls’s own philosophical views, is a good enough criticism of his original theory, but I will add at the end what I think are the strongest critiques I know."

Hazlett's Political Spectrum by Robin Hanson - "Not only would everything have been much better without FCC regulation, it actually was much better before the FCC! Herbert Hoover, who was head of the US Commerce Department at the time, broke the spectrum in order to then “save” it, a move that probably helped him rise to the presidency."

Another Point Of View by Simon Penner (Status 451) - The author was raised working class in semi-rural Canada and moved to Silicon valley. He experienced a ton of culture shock and significant cultural discrimination. This causes him to have less sympathy for people who quit software because of relatively minor pressure saying they don't fit in. The author overcame this stuff and so should other people.

Rapid Onset Gender Dysphoria Is Bad Science by Ozy (Thing of Things) - Ozy cites two commonly misinterpreted but good studies about suicide rates among transgender individuals. Ozy then discusses a very shoddy study about people rapidly becoming trans after meeting trans friends. However the study got its information by asking the parents of trans teens and young adults. Ozy explains how and why young adults hide much of their feelings from their parents, especially if they are neurodiverse.

Housing Price Bubble Revisited by Tyler Cowen - "Over the entire 20th century real home prices averaged an index value of about 110 (and were quite close to this value over the the entire 1950-1997 period). Over the entire 20th century, housing prices never once roce above 131, the 1989 peak. But beginning around 2000 house prices seemed to reach for an entirely new equilibrium. In fact, even given the financial crisis, prices since 2000 fell below the 20th century peak for only a few months in late 2011. Real prices today are now back to 2004 levels and rising. As I predicted in 2008, prices never returned to their long-run 20th century levels."

Reinventing The Wheel Of Fortune by sam[]zdat - Two definitions of democracy. A key idea: "Lasch is an external commentary using this rough model. At some point, the combined apparatus of American culture (the state, capital, media, political agitation) tried to make things “better”. To better its citizens required new social controls (paternalism). The taylorism employed makes things more focused on image, and this results in a more warlike society. Happened with “authenticity” last time and also [everything below]. To deal with this invasion, society turns to narcissistic defenses. Narcissism is self-centered, but it’s an expression of dependence on others, and specifically on the others’ validation of the narcissist’s image."


Prime Towers Problem by protokol2020 - Prime height towers. From which tower is the most tower tops visible.

The Unyoga Manifesto by SquirrelInHell - Yoga has a sort of 'competitive' ethos baked in. There is alot of pressure to do the postures 'correctly'. Instead you should listen to your body and follow the natural incentive gradients that lead to maintaining one's body well. Four practical pieces of advice.

Clojure The Perfect Language To Expand Your Brain by Eli Bendersky - Clojure will almost certainly change how you think about programming. Clojure is a fully modern and useable Lisp. The designers of Clojure are extremely pragmatic, building upon decades of industry experience. Sequences and laziness for powerful in-language data processing. Right approach to OOP. Built-in support for concurrency and parallelism.

A Physics Problem Once Again by protokol2020 - Discussion of n-dimensional mating. Approximate the sum of all gravitational forces between pairs of atoms inside the earth.

Meta Contrarian Typography by Tom Bartleby - The author is a self-described meta-contrarian. Supporting two spaces after a period. The three reasons for single spaces and why they don't hold up. Double spaces makes writing easier to skim, periods are over-worked in English.

I Cant Be Your Hero Im Too Busy Being Super by Jim Stone (ribbonfarm) - "But people don’t generally take on the burdens of inauthenticity without good reason. Often it’s because they want to occupy social roles that allow them to get their physical and psychological needs met, and other people won’t let them play those roles unless they are the right kind of person. Sometimes people put on masks simply to secure the role of “community member” or “citizen” or “human being”."


Physical Training Dating Strategies And Stories From The Early Days by Tim Feriss - Tim answers viewer questions. Physical training, interview prep, the art of networking, education reform, dream guests on the show.

Living With Violence by Waking Up with Sam Harris - "Gavin de Becker about the primacy of human intuition in the prediction and prevention of violence."

Amanda Askell On Pascals Wager And Other Low Risks Wi by Rational Speaking - Pascal's Wager: It's rational to believe in God, because if you're wrong it's no big deal, but if you're right then the payoff is huge. Amanda Askell argues that it's much trickier to rebut Pascal's Wager than most people think. Handling low probability but very high impact possibilities: should you round them down to zero? Does it matter how measurable the risk is? And should you take into account the chance you're being scammed?"

Tyler Cowen On Stubborn Attachments by EconTalk - "Cowen argues that economic growth--properly defined--is the moral key to maintaining civilization and promoting human well-being. Along the way, the conversation also deals with inequality, environmental issues, and education"

40 Making Humans Legible by The Bayesian Conspiracy - Seeing like a State. Scott and Sam[]zdat's posts. Green Revolution. Age of Em. Chemtrails and invasive species. Friendship is Optimal.

Dave Rubin by Tyler Cowen - "Comedy and political correctness, which jokes should not be told, the economics of comedy, comedy in Israel and Saudi Arabia, comedy on campus, George Carlin, and the most underrated Star Wars installment"

Yascha Mounk by The Ezra Klein Show - Trump's illiberalism is catalyzed by his failures. Recently Trump has been more illiberal. Support for Trump remains at around 40 percent. What does this imply about the risk of an illiberal Trump successor with more political competence.

Alex Guarnasche by EconTalk - Food network star. "What it's like to run a restaurant, the challenges of a career in cooking, her favorite dishes, her least favorite dishes, and what she cooked to beat Bobby Flay."

On Becoming A Better Person by Waking Up with Sam Harris - "David Brooks. His book The Road to Character, the importance of words like “sin” and "virtue," self-esteem vs. self-overcoming, the significance of keeping promises, honesty, President Trump."

Julia Galef On How To Argue Better And Change Your Mind More by The Ezra Klein Show - Thinking more clearly and arguing better, Ezra's concerns that the traditional paths toward a better discourse. Signaling is turtles all the way down, motivated reasoning, probabilistic debating, which identities help us find truth, making online arguments less terrible. Julia heavily emphasizes the importance of good epistemic communities. Being too charitable can produce wrong predictions. Seeing like a State.

[Link] S-risks: Why they are the worst existential risks, and how to prevent them

19 Kaj_Sotala 20 June 2017 12:34PM

Rescuing the Extropy Magazine archives

18 Deku-shrub 01 July 2017 02:25PM

Possibly of more interest to old school Extropians, you may be aware the defunct Extropy Institute's website is very slow and broken, and certainly inaccessible to newcomers.

Anyhow, I have recently pieced together most of the early publications, 1988 - 1996 of 'Extropy: Vaccine For Future Shock' later, Extropy: Journal of Transhumanist Thought, as a part of mapping the history of Extropianism.

You'll find some really interesting very early articles on neural augmentation, transhumanism, libertarianism, AI (featuring Eliezer), radical economics (featuring Robin Hanson of course) and even decentralised payment systems.

Along with the ExI mailing list which is not yet wikified, it provides a great insight into early radical technological thinking, an era mostly known for the early hacker movement.

Let me know your thoughts/feedback!

Notes from the Hufflepuff Unconference (Part 1)

17 Raemon 23 May 2017 09:04PM

April 28th, we ran the Hufflepuff Unconference in Berkeley, at the MIRI/CFAR office common space.

There's room for improvement in how the Unconference could have been run, but it succeeded the core things I wanted to accomplish: 

 - Established common knowledge of what problems people were actually interested in working on
 - We had several extensive discussions of some of those problems, with an eye towards building solutions
 - Several people agreed to work together towards concrete plans and experiments to make the community more friendly, as well as build skills relevant to community growth. (With deadlines and one person acting as project manager to make sure real progress was made)
 - We agreed to have a followup unconference in roughly three months, to discuss how those plans and experiments were going

Rough notes are available here. (Thanks to Miranda, Maia and Holden for takin really thorough notes)

This post will summarize some of the key takeaways, some speeches that were given, and my retrospective thoughts on how to approach things going forward.

But first, I'd like to cover a question that a lot of people have been asking about:

What does this all mean for people outside of the Bay?

The answer depends.

I'd personally like it if the overall rationality community got better at social skills, empathy, and working together, sticking with things that need sticking with (and in general, better at recognizing skills other than metacognition). In practice, individual communities can only change in the ways the people involved actually want to change, and there are other skills worth gaining that may be more important depending on your circumstances.

Does Project Hufflepuff make sense for your community?

If you're worried that your community doesn't have an interest in any of these things, my actual honest answer is that doing something "Project Hufflepuff-esque" probably does not make sense. I did not choose to do this because I thought it was the single-most-important thing in the abstract. I did it because it seemed important and I knew of a critical mass of people who I expected to want to work on it. 

If you're living in a sparsely populated area or haven't put a community together, the first steps do not look like this, they look more like putting yourself out there, posting a meetup on Less Wrong and just *trying things*, any things, to get something moving.

If you have enough of a community to step back and take stock of what kind of community you want and how to strategically get there, I think this sort of project can be worth learning from. Maybe you'll decide to tackle something Project-Hufflepuff-like, maybe you'll find something else to focus on. I think the most important thing is have some kind of vision for something you community can do that is worth working together, leveling up to accomplish.

Community Unconferences as One Possible Tool

Community unconferences are a useful tool to get everyone on the same page and spur them on to start working on projects, and you might consider doing something similar. 

They may not be the right tool for you and your group - I think they're most useful in places where there's enough people in your community that they don't all know each other, but do have enough existing trust to get together and brainstorm ideas. 

If you have a sense that Project Hufflepuff is worthwhile for your community but the above disclaimers point towards my current approach not making sense for you, I'm interested in talking about it with you, but the conversation will look less like "Ray has ideas for you to try" and more like "Ray is interested in helping you figure out what ideas to try, and the solution will probably look very different."

Online Spaces

Since I'm actually very uncertain about a lot of this and see it as an experiment, I don't think it makes sense to push for any of the ideas here to directly change Less Wrong itself (at least, yet). But I do think a lot of these concepts translate to online spaces in some fashion, and I think it'd make sense to try out some concepts inspired by this in various smaller online subcommunities.

Table of Contents:

I. Introduction Speech

 - Why are we here?
 - The Mission: Something To Protect
 - The Invisible Badger, or "What The Hell Is a Hufflepuff?"
 - Meta Meetups Usually Suck. Let's Try Not To.

II. Common Knowledge

 - What Do People Actually Want?
 - Lightning Talks

III. Discussing the Problem (Four breakout sessions)

 - Welcoming Newcomers
 - How to handle people who impose costs on others?
 - Styles of Leadership and Running Events
 - Making Helping Fun (or at least lower barrier-to-entry)

IV. Planning Solutions and Next Actions

V. Final Words

I. Introduction: It Takes A Village to Save a World

(A more polished version of my opening speech from the unconference)

[Epistemic Status: This is largely based on intuition, looking at what our community has done and what other communities seem to be able to do. I'm maybe 85% confident in it, but it is my best guess]

In 2012, I got super into the rationality community in New York. I was surrounded by people passionate about thinking better and using that thinking to tackle ambitious projects. And in 2012 we all decided to take on really hard projects that were pretty likely to fail, because the expected value seemed high, and it seemed like even if we failed we'd learn a lot in the process and grow stronger.

That happened - we learned and grew. We became adults together, founding companies and nonprofits and creating holidays from scratch.

But two years later, our projects were either actively failing, or burning us out. Many of us became depressed and demoralized.

There was nobody who was okay enough to actually provide anyone emotional support. Our core community withered.

I ended up making that the dominant theme of the 2014 NYC Solstice, with a call-to-action to get back to basics and take care each other.

I also went to the Berkeley Solstice that year. And... I dunno. In the back of my mind I was assuming "Berkeley won't have that problem - the Bay area has so many people, I can't even imagine how awesome and thriving a community they must have." (Especially since the Bay kept stealing all the Movers and Shakers of NYC).

The theme of the Bay Solstice turned out to be "Hey guys, so people keep coming to the Bay, running on a dream and a promise of community, but that community is not actually there, there's a tiny number of well-connected people who everyone is trying to get time with, and everyone seems lonely and sad. And we don't even know what to do about this."

In 2015, that theme in the Berkeley Solstice was revisited.

So I think that was the initial seed of what would become Project Hufflepuff - noticing that it's not enough to take on cool projects, that it's not enough to just get a bunch of people together and call it a community. Community is something you actively tend to. Insofar as Maslow's hierarchy is real, it's a foundation you need before ambitious projects can be sustainable.

There are other pieces of the puzzle - different lenses that, I believe, point towards a Central Thing. Some examples:

Group houses, individualism and coordination.

I've seen several group houses where, when people decide it no longer makes sense to live in the house, they... just kinda leave. Even if they've literally signed a lease. And everyone involved (the person leaving and those remain), instinctively act as if it's the remaining people's job to fill the leaver's spot, to make rent.

And the first time, this is kind of okay. But then each subsequent person leaving adds to a stressful undertone of "OMG are we even going to be able to afford to live here?". It eventually becomes depressing, and snowballs into a pit that makes newcomers feel like they don't WANT to move into the house.

Nowadays I've seen some people explicitly building into the roommate agreement a clear expectation of how long you stay and who's responsibility it is to find new roommates and pay rent in the meantime. But it's disappointing to me that this is something we needed, that we weren't instinctively paying to attention to how we were imposing costs on each other in the first place. That when we *violated a written contract*, let alone a handshake agreement, that we did not take upon ourselves (or hold each other accountable), to ensure we could fill our end of the bargain.

Friends, and Networking your way to the center

This community puts pressure on people to improve. It's easier to improve when you're surrounded by ambitious people who help or inspire each other level up. There's a sense that there's some cluster of cool-people-who-are-ambitious-and-smart who've been here for a while, and... it seems like everyone is trying to be friends with those people. 

It also seems like people just don't quite get that friendship is a skill, that adult friendships in City Culture can be hard, and it can require special effort to make them happen.

I'm not entirely sure what's going on here - it doesn't make sense to say anyone's obligated to hang out with any particular person (or obligated NOT to), but if 300 people aren't getting the connection they want it seems like *somewhere people are making a systematic mistake.* 

(Since the Unconference, Maia has tackled this particular issue in more detail)


The Mission - Something To Protect


As I see it, the Rationality Community has three things going on: Truth. Impact. And "Being People".

In some sense, our core focus is the practice of truthseeking. The thing that makes that truthseeking feel *important* is that it's connected to broader goals of impacting the world. And the thing that makes this actually fun and rewarding enough to stick with is a community that meets our needs, where can both flourish as individuals and find the relationships we want.

I think we have made major strides in each of those areas over the past seven years. But we are nowhere near done.

Different people have different intuitions of which of the three are most important. Some see some of them as instrumental, or terminal. There are people for whom Truthseeking is *the point*, and they'd have been doing that even if there wasn't a community to help them with it, and there are people for whom it's just one tool of many that helps them live their life better or plan important projects.

I've observed a tendency to argue about which of these things is most important, or what tradeoffs are worth making. Inclusiveness verses high standards. Truth vs action. Personal happiness vs high acheivement.

I think that kind of argument is a mistake.

We are falling woefully short on all of these things. 

We need something like 10x our current capacity for seeing, and thinking. 10x our capacity for doing. 10x our capacity for *being healthy people together.*

I say "10x" not because all these things are intrinsically equal. The point is not to make a politically neutral push to make all the things sound nice. I have no idea exactly how far short we're falling on each of these because the targets are so far away I can't even see the end, and we are doing a complicated thing that doesn't have clear instructions and might not even be possible.

The point is that all of these are incredibly important, and if we cannot find a way to improve *all* of these, in a way that is *synergistic* with each other, then we will fail.

There is a thing at the center of our community. Not all of us share the exact same perspective on it. For some of us it's not the most important thing. But it's been at the heart of the community since the beginning and I feel comfortable asserting that it is the thing that shapes our culture the most:

The purpose of our community is to make sure this place is okay:

The world isn't okay right now, on a number of levels. And a lot of us believe there is a strong chance it could become dramatically less okay. I've seen people make credible progress on taking responsibility for pieces of our home. But when all is said and done, none of our current projects really give me the confidence that things are going to turn out all right. 

Our community was brought together on a promise, a dream, and we have not yet actually proven ourselves worthy of that dream. And to make that dream a reality we need a lot of things.

We need to be able to criticize, because without criticism, we cannot improve.

If we cannot, I believe we will fail.

We need to be able to talk about ideas that are controversial, or uncomfortable - otherwise our creativity and insight will be crippled.

If we cannot, I believe we will fail.

We need to be able to do those things without alienating people. We need to be able to criticize without making people feel untrusted and discouraged from even taking action. We need to be able to discuss challenging things while earnestly respecting the notion that *talking about ideas gives those ideas power and has concrete effects on social reality*, and sometimes that can hurt people.

If we cannot figure out how to do that, I believe we will fail.

We need more people who are able and willing to try things that have never been done before. To stick with those things long enough to *get good at them*, to see if they can actually work. We need to help each other do impossible things. And we need to remember to check for and do the *possible*, boring, everyday things that are in fact straightforward and simple and not very inspiring. 

If we cannot manage to do that, I believe we will fail.

We need to be able to talk concretely about what the *highest leverage actions in the world are*. We need to prioritize those things, because the world is huge and broken and we are small. I believe we need to help each other through a long journey, building bigger and bigger levers, building connections with people outside our community who are undertaking the same journey through different perspectives.

And in the process, we need to not make it feel like if *you cannot personally work on those highest leverage things, that you are not important.* 

There's the kind of importance where we recognize that some people have scarce skills and drive, and the kind of importance where we remember that *every* person has intrinsic worth, and you owe *nobody* any special skills or prestigious sounding projects for your life to be worthwhile.

This isn't just a philosophical matter - I think it's damaging to our mental health and our collective capacity. 

We need to recognize that the distribution of skills we tend to reward or punish is NOT just about which ones are actually most valuable - sometimes it is simply founder effects and blind spots.

We cannot be a community for everyone - I believe trying to include anyone with a passing interest in us is a fool's errand. But there are many people who had valuable skills to contribute who have turned away, feeling frustrated and un-valued.

If we cannot find a way to accomplish all of these things at once, I believe we will fail.

The thesis of Project Hufflepuff is that it takes (at least) a village to save a world. 

It takes people doing experimental impossible things. It takes caretakers. It takes people helping out with unglorious tasks. It takes technical and emotional and physical skills. And while it does take some people who specialize in each of those things, I think it also needs many people who are least a little bit good at each of them, to pitch in when needed.

Project Hufflepuff is not the only things our community needs, or the most important. But I believe it is one of the necessary things that our community needs, if we're to get to 10x our current Truthseeking, Impact and Human-ing.

If we're to make sure that our home is okay.

The Invisible Badger

"A lone hufflepuff surrounded by slytherins will surely wither as if being leeched dry by vampires."

- Duncan

[Epistemic Status: My evidence for this is largely based on discussions with a few people for whom the badger seems real and valuable, and who report things being different in other communities, as well as some of my general intuitions about society. I'm 75% sure the badger exists, 90% that's it worth leaning into the idea of the badger to see if it works for you, and maybe 55% sure that it's worth trying to see the badger if you can't already make out it's edges.]


If I *had* to pick a clear thing that this conference is about without using Harry Potter jargon, I'd say "Interpersonal dynamics surrounding trust, and how those dynamics apply to each of the Impact/Truth/Human focuses of the rationality community."

I'm not super thrilled with that term because I think I'm grasping more for some kind of gestalt. An overall way of seeing and being that's hard to describe and that doesn't come naturally to the sort of person attracted to this community.

Much like the blind folk and the elephant, who each touched a different part of the animal and came away with a different impression (the trunk seems like a snake, the legs seem like a tree), I've been watching several people in the community try to describe things over the past few years. And maybe those things are separate but I feel like they're secretly a part of the same invisible badger.

Hufflepuff is about hard work, and loyalty, and camaraderie. It's about emotional intelligence. It's about seeing value in day to day things that don't directly tie into epic narratives. 

There's a bunch of skills that go into Hufflepuff. And part of want I want is for people to get better at those skills. But It think a mindset, an approach, that is fairly different from the typical rationalist mindset, that makes those skills easier. It's something that's harder when you're being rigorously utilitarian and building models of the world out of game theory and incentives.

Mindspace is deep and wide, and I don't expect that mindset to work for everyone. I don't think everyone should be a Hufflepuff. But I do think it'd be valuable to the community if more people at least had access to this mindset and more of these skills.

So what I'd like, for tonight, is for people to lean into this idea. Maybe in the end you'll find that this doesn't work for you. But I think many people's first instinct is going to be that this is alien and uncomfortable and I think it's worth trying to push past that.

The reason we're doing this conference together is because the Hufflepuff way doesn't really work if people are trying to do it alone - I think it requires trust and camaraderie and persistence to really work. I don't think we can have that required trust all at once, but I think if there are multiple people trying to make it work, who can incrementally trust each other more, I think we can reach a place where things run more smoothly, where we have stronger emotional connections, and where we trust each other enough to take on more ambitious projects than we could if we're all optimizing as individuals.

Meta-Meetups Suck. Let's Not.

This unconference is pretty meta - we're talking about norms and vague community stuff we want to change.

Let me tell you, meta meetups are the worst. Typically you end up going around in circles complaining and wishing there were more things happening and that people were stepping up and maybe if you're lucky you get a wave of enthusiasm that lasts a month or so and a couple things happen but nothing really *changes*.

So. Let's not do that. Here's what I want to accomplish and which seems achievable:

1) Establish common knowledge of important ideas and behavior patterns. 

Sometimes you DON'T need to develop a whole new skill, you just need to notice that your actions are impacting people in a different way, and maybe that's enough for you to decide to change somethings. Or maybe someone has a concept that makes it a lot easier for you to start gaining a new skill on your own.

2) Establish common knowledge of who's interested in trying which new norms, or which new skills. 

We don't actually *know* what the majority of people want here. I can sit here and tell you what *I* think you should want, but ultimately what matters is what things a critical mass of people want to talk about tonight.

Not everyone has to agree that an idea is good to try it out. But there's a lot of skills or norms that only really make sense when a critical mass of other people are trying them. So, maybe of the 40 people here, 25 people are interested in improving their empathy, and maybe another 20 are interested in actively working on friendship skills, or sticking to commitments. Maybe those people can help reinforce each other.

3) Explore ideas for social and skillbuilding experiments we can try, that might help. 

The failure mode of Ravenclaws is to think about things a lot and then not actually get around to doing them. A failure mode of ambitious Ravenclaws, is to think about things a lot and then do them and then assume that because they're smart, that they've thought of everything, and then not listen to feedback when they get things subtly or majorly wrong.

I'd like us to end by thinking of experiments with new norms, or habits we'd like to cultivate. I want us to frame these as experiments, that we try on a smaller scale and maybe promote more if they seem to be working, while keeping in mind that they may not work for everyone.

4) Commit to actions to take.

Since the default action is for them to peter out and fail, I'd like us to spend time bulletproofing them, brainstorming and coming up with trigger-action plans so that they actually have a chance to succeed.

Tabooing "Hufflepuff"

Having said all that talk about The Hufflepuff Way...

...the fact is, much of the reason I've used those towards is to paint a rough picture to attract the sort of person I wanted to attract to this unconference.

It's important that there's a fuzzy, hard-to-define-but-probably-real concept that we're grasping towards, but it's also important not to be talking past each other. Early on in this project I realized that a few people who I thought were on the same page actually meant fairly different things. Some cared more about empathy and friendship. Some cared more about doing things together, and expected deep friendships to arise naturally from that.

So I'd like us to establish a trigger-action-plan right now - for the rest of this unconference, if someone says "Hufflepuff", y'all should say "What do you mean by that?" and then figure out whatever concrete thing you're actually trying to talk about.

II. Common Knowledge

The first part of the unconference was about sharing our current goals, concerns and background knowledge that seemed useful. Most of the specifics are covered in the notes. But I'll talk here about why I included the things I did and what my takeaways were afterwards on how it worked.

Time to Think

The first thing I did was have people sit and think about what they actually wanted to get out of the conference, and what obstacles they could imagine getting in the way of that. I did this because often, I think our culture (ostensibly about helping us think better) doesn't give us time to think, and instead has people were are quick-witted and conversationally dominant end up doing most of the talking. (I wrote a post a year ago about this, the 12 Second Rule). In this case I gave everyone 5 minutes, which is something I've found helpful at small meetups in NYC.

This had mixed results - some people reported that while they can think well by themselves, in a group setting they find it intimidating and their mind starts wandering instead of getting anything done. They found it much more helpful when I eventually let people-who-preferred-to-talk-to-each-other go into another room to talk through their ideas outloud.

I think there's some benefit to both halves of this and I'm not sure how common which set of preferences are. It's certainly true that it's not common for conferences to give people a full 5 minutes to think so I'd expect it to be someone uncomfortable-feeling regardless of whether it was useful.

But an overall outcome of the unconference was that it was somewhat lower energy than I'd wanted, and opening with 5 minutes of silent thinking seemed to contribute to that, so for the next unconference I run, I'm leaning towards a shorter period of time for private thinking (Somewhere between 12 and 60 seconds), followed by "turn to your neighbors and talk through the ideas you have", followed by "each group shares their concepts with the room."

"What is do you want to improve on? What is something you could use help with?"

I wanted people to feel like active participants rather than passive observers, and I didn't want people to just think "it'd be great if other people did X", but to keep an internal locus of control - what can *I* do to steer this community better? I also didn't want people to be thinking entirely individualistically.

I didn't collect feedback on this specific part and am not sure how valuable others found it (if you were at the conference, I'd be interested if you left any thoughts in the comments). Some anonymized things people described:

  • When I make social mistakes, consider it failure; this is unhelpful

  • Help point out what they need help with

  • Have severe akrasia, would like more “get things done” magic tools

  • Getting to know the bay area rationalist community

  • General bitterness/burned out

  • Reduce insecurity/fear around sharing

  • Avoiding spending most words signaling to have read a particular thing; want to communicate more clearly

  • Creating systems that reinforce unnoticed good behaviour

  • Would like to learn how to try at things

  • Find place in rationalist community

  • Staying connected with the group

  • Paying attention to what they want in the moment, in particular when it’s right to not be persistent

  • Would like to know the “landing points” to the community to meet & greet new people

  • Become more approachable, & be more willing to approach others for help; community cohesiveness

  • Have been lonely most of life; want to find a place in a really good healthy community

  • Re: prosocialness, being too low on Maslow’s hierarchy to help others

  • Abundance mindset & not stressing about how to pay rent

  • Cultivate stance of being able to do helpful things (action stance) but also be able to notice difference between laziness and mental health

  • Don’t know how to respect legit safety needs w/o getting overwhelmed by arbitrary preferences; would like to model people better to give them basic respect w/o having to do arbitrary amount of work

  • Starting conversations with new people

  • More rationalist group homes / baugruppe

  • Being able to provide emotional support rather than just logistics help

  • Reaching out to people at all without putting too much pressure on them

  • Cultivate lifelong friendships that aren’t limited to particular time and place

  • Have a block around asking for help bc doesn’t expect to reciprocate; would like to actually just pay people for help w stuff

  • Want to become more involved in the community

  • Learn how to teach other people “ops skills”

  • Connections to people they can teach and who can teach them

Lightning Talks

Lightning talks are a great way to give people an opportunity to not just share ideas, but get some practice at public presentation (which I've found can be a great gateway tool for overall confidence and ability to get things done in the community). Traditionally they are 5 minutes long. CFAR has found that 3.5 minute lightning talks are better than 5 minute talks, because it cuts out some rambling and tangents.

It turned out we had more people than I'd originally planned time for, so we ended up switching to two minute talks. I actually think this was even better, and my plan for next time is do 1-minute timeslots but allow people to sign up for multiple if they think their talk requires it, so people default to giving something short and sweet.

Rough summaries of the lightning talks can be found in the notes.

III. Discussing the Problem

The next section involved two "breakout session" - two 20 minute periods for people to split into smaller groups and talk through problems in detail. This was done in an somewhat impromptu fashion, with people writing down the talks they wanted to do on the whiteboard and then arranging them so most people could go to a discussion that interested them.

The talks were:

 -  Welcoming Newcomers
 -  How to handle people who impose costs on others?
 -  Styles of Leadership and Running Events
 -  Making Helping Fun (or at least lower barrier-to-entry)
 -  Circling session 

There was a suggested discussion about outreach, which I asked to table for a future unconference. My reason was that outreach discussions tend to get extremely meta and seem to be an attractor (people end up focusing on how to bring more people into the community without actually making sure the community is good, and I wanted the unconference to focus on the latter.)

I spent some time drifting between sessions, and was generally impressed both with the practical focus each discussion had, as well as the way they were organically moderated.

Again, more details in the notes.

IV. Planning Solutions and Next Actions

After about an hour of discussion and mingling, we came back to the central common space to describe key highlights from each session, and begin making concrete plans. (Names are crediting people who suggested an idea and who volunteered to make it happen)

Creating Norms for Your Space (Jane Joyce, Tilia Bell)

The "How to handle people who impose costs on other" conversation ended up focusing on minor but repeated costs. One of the hardest things to moderate as an event host is not people who are actively disruptive, but people who just a little bit awkward or annoying - they'd often be happy to change their behavior if they got feedback, but giving feedback feels uncomfortable and it's hard to do in a tactful way. This presents two problems at once: parties/events/social-spaces end up a more awkward/annoying than they need to be, and often what happens is that rather than giving feedback, the hosts stop inviting people doing those minor things, which means a lot of people still working on their social skills end up living in fear of being excluded.

Solving this fully requires a few different things at once, and I'm not sure I have a clear picture of what it looks like, but one stepping stone people came up with was creating explicit norms for a given space, and a practice of reminding people of those norms in a low-key, nonjudgmental way.

I think will require a lot of deliberate effort and practice on the part of hosts to avoid alternate bad outcomes like "the norms get disproportionately enforced on people the hosts like and applied unfairly to people they aren't close with". But I do think it's a step in the right direction to showcase what kind of space you're creating and what the expectations are.

Different spaces can be tailored for different types of people with different needs or goals. (I'll have more to say about this in an upcoming post - doing this right is really hard, I don't actually know of any groups that have done an especially good job of it.)

I *was* impressed with the degree to which everyone in the conversation seemed to be taking into account a lot of different perspectives at once, and looking for solutions that benefited as many people as possible.

Welcoming Committee (Mandy Souza, Tessa Alexanian)

Oftentimes at events you'll see people who are new, or who don't seem comfortable getting involved with the conversation. Many successful communities do a good job of explicitly welcoming those people. Some people at the unconference decided to put together a formal group for making sure this happens more.

The exact details are still under development, but I think the basic idea is to have a network of people who are interested
he idea is to have a group of people who go to different events, playing the role of the welcomer. I think the idea is sort of a "Uber for welcomers" network (i.e. it both provides a place for people running events to go to ask for help with welcoming, and people who are interested in welcoming to find events that need welcomers)

It also included some ideas for better infrastructure, such as reviving "" to make it easier for newcomers to figure out what events are going on (possibly including links to the codes of conduct for different spaces as well). In the meanwhile, some simple changes were the introduction of a facebook group for Bay Area Rationalist Social Events.

Softskill-sharing Groups (Mike Plotz and Jonathan Wallis)

The leadership styles discussion led to the concept that in order to have a flourishing community, and to be a successful leader, it's valuable to make yourself legible to others, and others more legible to yourself. Even small improvements in an activity as frequent as communication can have huge effects over time, as we make it easier to see each other as we actually are and to clearly exchange our ideas. 

A number of people wanted to improve in this area together, and so we’re working towards establishing a series of workshops with a focus on practice and individual feedback. A longer post on why this is important is coming up, and there will be information on the structure of the event after our first teacher’s meeting. If you would like to help out or participate, please fill out this poll:

Circling Explorations (Qiaochu and others)

Much of the discussion at the Unconference, while focused on community, ultimately was explored through an intellectual lens. By contrast, "Circling" is a practice developed by the Authentic Relating community which is focused explicitly on feelings. The basic premise is (sort of) simple: you sit in a circle in a secluded space, and you talk about how you're feeling in the moment. Exactly how this plays out is a bit hard to explain, but the intended result is to become better both at noticing your own feelings and the people around you.

Opinions were divided as to whether this was something that made sense for "rationalists to do on their own", or whether it made more sense to visit more explicitly Circling-focused communities, but several people expressed interest in trying it again.

Making Helping Fun and More Accessible (Suggested by Oliver Habryka)

Ultimately we want a lot of people who are able and excited to help out with challenging projects - to improve our collective group ambition. But to get there, it'd be really helpful to have "gateway helping" - things people can easily pitch in to do that are fun, rewarding, clearly useful but on the "warm fuzzies" side of helping. Oliver suggested this as a way to get people to start identifying as people-who-help.

There were two main sets of habits that worth cultivating:

1) Making it clear to newcomers that they're encouraged to help out with events, and that this is actually a good way to make friends and get more involved. 

2) For hosts and event planners, look for opportunities to offer people things that they can help with, and make sure to publicly praise those who do help out.

Some of this might dovetail nicely with the Welcoming Committee, both as something people can easily get involved with, and if there ends up being a public facing website to introduce people to the community, using that to connect people with events that could use help).

Volunteering-as-Learning, and Big Event Specific Workshops

Sometimes volunteering just requires showing up. But sometimes it requires special skills, and some events might need people who are willing to practice beforehand or learn-by-doing with a commitment to help at multiple events.

A vague cluster of skills that's in high demand is "predict logistical snafus in advance to head them off, and notice logistical snafus happening in realtime so you can do something about them." Earlier this year there was an Ops Workshop that aimed to teach this sort of skill, which went reasonably but didn't really lead into a concrete use for the skills to help them solidify.

One idea was to do Ops workshops (or other specialized training) in the month before a major event like Solstice or EA Global, giving them an opportunity to practice skills and making that particular event run smoother.

(This specific idea is not currently planned for implementation as it was among the more ambitious ones, although Brent Dill's series of "practice setting up a giant dome" beach parties in preparation for Burning Man are pointing in a similar direction)

Making Sure All This Actually Happens (Sarah Spikes, and hopefully everyone!)

To avoid the trap of dreaming big and not actually getting anything done, Sarah Spikes volunteered as project manager, creating an Asana page. People who were interested in committing to a deadline could opt into getting pestered by her to make sure things things got done. 

V. Parting Words

To wrap up the event, I focused on some final concepts that underlie this whole endeavor. 

The thing we're aiming for looks something like this:

In a couple months (hopefully in July), there'll be a followup unconference. The theme will be "Innovation and Excellence", addressing the twofold question "how do we encourage more people to start cool projects", and "how to do we get to a place where longterm projects ultimately reach a high quality state?"

Both elements feel important to me, and they require somewhat different mindsets (both on the part of the people running the projects, and the part of the community members who respond to them). Starting new things is scary and having too high standards can be really intimidating, yet for longterm projects we may want to hold ourselves to increasingly high standards over time.

My current plan (subject to lots of revision) is for this to become a series of community unconferences that happen roughly every 3 months. The Bay area is large enough with different overlapping social groups that it seems worthwhile to get together every few months and have an open-structured event to see people you don't normally see, share ideas, and get on the same page about important things.

Current thoughts for upcoming unconference topics are:

Innovation and Excellence
Personal Epistemic Hygiene
Group Epistemology

An important piece of each unconference will be revisiting things at the previous one, to see if projects, ideas or experiments we talked about were actually carried out and what we learned from them (most likely with anonymous feedback collected beforehand so people who are less comfortable speaking publicly have a chance to express any concerns). I'd also like to build on topics from previous unconferences so they have more chance to sink in and percolate (for example, have at least one talk or discussion about "empathy and trust as related to epistemic hygiene").

Starting and Finishing Unconferences Together

My hope is to get other people involved sooner rather than later so this becomes a "thing we are doing together" rather than a "thing I am doing." One of my goals with this is also to provide a platform where people who are interested in getting more involved with community leadership can take a step further towards that, no matter where they currently stand (ranging anywhere from "give a 30 second lightning talk" to "run a discussion, or give a keynote talk" to "be the primary organizer for the unconference.")

I also hope this is able to percolate into online culture, and to other in-person communities where a critical mass of people think this'd be useful. That said, I want to caution that I consider this all an experiment, motivated by an intuitive sense that we're missing certain things as a culture. That intuitive sense has yet to be validated in any concrete fashion. I think "willingness to try things" is more important than epistemic caution, but epistemic caution is still really important - I recommend collecting lots of feedback and being willing to shift direction if you're trying anything like the stuff suggested here.

(I'll have an upcoming post on "Ways Project Hufflepuff could go horribly wrong")

Most importantly, I hope this provides a mechanism for us to collectively take ideas more seriously that we're ostensibly supposed to be taking seriously. I hope that this translates into the sort of culture that The Craft and The Community was trying to point us towards, and, ideally, eventually, a concrete sense that our community can play a more consistently useful role at making sure the world turns out okay. 

If you have concerns, criticism, or feedback, I encourage you to comment here if you feel comfortable, or on the Unconference Feedback Form. So far I've been erring on the side of move forward and set things in motion, but I'll be shifting for the time being towards "getting feedback and making sure this thing is steering in the right direction."


In addition to the people listed throughout the post, I'd like to give particular thanks to Duncan Sabien for general inspiration and a lot of concrete help, Lahwran for giving the most consistent and useful feedback, and Robert Lecnik for hosting the space. 

[Link] How I [Kaj] found & fixed the root problem behind my depression and anxiety after 20+ years

16 Kaj_Sotala 26 July 2017 12:56PM

Bi-Weekly Rational Feed

16 deluks917 28 May 2017 05:12PM

Five Recommended Articles You Might Have Missed:

The Four Blind Men The Elephant And Alan Kay by Meredith Paterson (Status 451) - Managing technical teams. Taking a new perspective is worth 90 IQ points. Getting better enemies. Guerrilla action.

Vast Empirical Literature by Marginal REVOLUTION - Tyler's 10 thoughts on approaching fields with large literatures. He is critical of Noah's "two paper rule" and recommends alot of reading.

Notes From The Hufflepuff Unconference (Part 1) by Raemon (lesswrong) - Goal: Improve at: "social skills, empathy, and working together, sticking with things that need sticking with". The article is a detailed breakdown of the unconference including: Ray's Introductory Speech, a long list of what people want to improve on, the lightning talks, the 4 breakout sessions, proposed solutions, further plans, and closing words. Links to conference notes are included for many sections.

Antipsychotics Might Cause Cognitive Impairment by Sarah Constantin (Otium) - A harrowing personal account of losing abstract thinking ability on Risperdal. The author conducts a literature review, and concludes with some personal advice about taking medication.

Dwelling In Possibility by Sarah Constantin (Otium) - Leadership. Confidence in the face of the uncertainty and imperfection. Losing yourself when you try to step back and facilitate.


Those Modern Pathologies by Scott Alexander - You can argue X is a modern pathology for almost any value of X. Scott demonstrates this by repeated example. Among other things "Aristotelian theory of virtue" and "Homer's Odyssey" get pathologized.

The Atomic Bomb Considered As Hungarian High School Science Fair Project by Scott Alexander - Ashkenazi Jewish Intelligence. An explanation of Hungarian dominance in physics and science in the mid 1900s.

Classified Ads Thread by Scott Alexander - Open thread where people post ads. People are promoting their websites and some of them are posting actual job ads among other things.

Open Thread 76 by Scott Alexander - Bi-weekly Open thread.

Postmarketing Surveillance Is Good And Normal by Scott Alexander - Scott shows why a recent Scientific American study does not imply the FDA is too risky.

Epilogue by Scott Alexander (Unsong) - All's Whale that Ends Whale.

Polyamory Is Not Polygyny by Scott Alexander - A quick review of how polyamory actually function in the rationalist community.

Bail Out by Scott Alexander - "About a fifth of the incarcerated population – the top of the orange slice, in this graph – are listed as “not convicted”. These are mostly people who haven’t gotten bail. Some are too much of a risk. But about 40% just can’t afford to pay."


Strong Men Are Socialist Reports A Study That Previously Reported The Opposite by Jacob Falkovich (Put A Number On It!) - Defense Against the Dark Statistical Arts. Jacob provides detailed commentary on a popular study and shows that the studies dataset can be used to support the opposite conclusion, with p = 0.0086.

Highly Advanced Tulpamancy 101 For Beginners by H i v e w i r e d - Application of lesswrong theory to the concept of the self. In particular the author applies "How an Algorithm Feels from the Inside" and "Map and Territory". Hive then goes into the details of creating and interacting with tulpas. "A tulpa is an autonomous entity existing within the brain of a “host”. They are distinct from the host in that they possess their own personality, opinions, and actions"

Existential Risk From Ai Without An Intelligence by Alex Mennen (lesswrong) - Reasons why an intelligence explosion might not occur and reasons why we might have a problem anyway.

Dragon Army Theory Charter (30min Read) by Duncan Sabien (lesswrong) - A detailed plan for an ambitious military style rationalist house. The major goals include self-improvement, high quality group projects and the creation of a group with absolute trust in one another. The leader of the house is the curriculum director and head of product at CFAR.

The Story Of Our Life by H i v e w i r e d - The authors explain their pre-rationalist life and connection to the community. They then argue the rationalist community should take better care of one another. "Venture Rationalism".

Don't Believe in God by Tyler Cowen - Seven arguments for not believing in God. Among them: Lack of Bayesianism among believers, the degree to which people follow their family religion and the fundamental weirdness of reality.

Antipsychotics Might Cause Cognitive Impairment by Sarah Constantin (Otium) - A harrowing personal account of losing abstract thinking ability on Risperdal. The author conducts a literature review, and concludes with some personal advice about taking medication.

The Four Blind Men The Elephant And Alan Kay by Meredith Paterson (Status 451) - Managing technical teams. Taking a new perspective is worth 90 IQ points. Getting better enemies. Guerrilla action.

Qualia Computing At Consciousness Hacking June 7th 2017 by Qualia Computing - Qualia computing will present in San Fransisco on June 7th at Consciousness Hacking. The event description is detailed and should give readers a good intro to Qualia Computing's goals. The author's research goal is to create a mathematical theory of pain/pleasure and be able to measure these directly from brain data.

Notes From The Hufflepuff Unconference (Part 1) by Raemon (lesswrong) - Goal: Improve at: "social skills, empathy, and working together, sticking with things that need sticking with". The article is a detailed breakdown of the unconference including: Ray's Introductory Speech, a long list of what people want to improve on, the lightning talks, the 4 breakout sessions, proposed solutions, further plans, and closing words. Links to conference notes are included for many sections.

Is Silicon Valley Real by Ben Hoffman (Compass Rose) - The old culture of Silicon Valley is mostly gone, replaced by something overpriced and materialist. Ben check's the details of Scott Alexander's list of six noble startups and finds only two in SV proper.

Why Is Harry Potter So Popular by Ozy (Thing of Things) - Ozy discusses a paper on song popularity in an artificial music market. Social dynamics had a big impact on song ratings. "Normal popularity is easily explicable by quality. Stupid, wild, amazing popularity is due to luck."

Design A Better Chess by Robin Hanson - Can we design a game that promotes even more useful honesty than chess? A link to Hanson's review of Gary Kasparov's book is included.

Deserving Truth 2 by Andrew Critch - How the author's values changed over time. Originally he tried to maximize his own positive sensory experiences. The things he cared about began to include more things, starting with his GF's experiences and values. He eventually rejects "homo-economus" thinking.

A Theory Of Hypocrisy by João Eira (Lettuce be Cereal) - Hypocrisy evolved as a way to solve free rider problems. "It pays to be a free rider. If no one finds out"

Building Community Institution In Five Hours a Week by Particular Virtue - Eight pieces of advice for running a successful meetup. The author and zir partner have been running lesswrong events for five years.

Dwelling In Possibility by Sarah Constantin (Otium) - Leadership. Confidence in the face of the uncertainty and imperfection. Losing yourself when you try to step back and facilitate.

Ai Safety Three Human Problems And One Ai Issue by Stuart Armstrong (lesswrong) - Humans have poor predictions, don't know their values and aren't agents. Ai might be very powerful. A graph of which problems many Ai risk solutions target.

Recovering From Failure by mindlevelup - Avoid negative spirals, figure out why you failed, List of questions to ask yourself. Strategies -> Generate good alternatives, metacognitive affordances.

Review The Dueling Neurosurgeons by Sam Kean by Aceso Under Glass - Positive review. Author learned alot. Speculation on a better way to teach Science.

Principia Qualia Part 2: Valence by Qualia Computing - A mathematical theory of valence (what makes experience feel good or bad). Speculative but the authors make concrete predictions. Music plays a heavy role.

Im Not Seaing It by Robin Hanson - Arguments against seasteading.


One of the more positive surprises by GiveDirectly - Links post. Eight articles on Give Directly, Cash Transfer and Basic Income.

Returns Functions And Funding Gaps by the Center for Effective Altruism (EA forum) - Links to CEA's explanation of what "returns functions" are and how using them compares to "funding gap" model. They give some arguments why returns functions are a superior model.

Online Google Hangout On Approaches To by whpearson (lesswrong) - Community meeting to discuss Ai risk. Will use "Optimal Brainstorming Theory". Currently early stage. Sign up and vote on what times you are available.

Expected Value Estimates We Cautiously Took by The Oxford Prioritization Project (EA forum) - Details of how the four bayesian probability models were compared to produce a final decision. Some discussion of how assumptions affect the final result. Actual code is included.

Four Quantitative Models Aggregation And Final by The Oxford Prioritization Project (EA forum) - 80K hours, MIRI, Good Foods Institute and StrongMinds were considered. Decisions were made using concrete Bayesian EV calculations. Links to the four models are included.

Peer to Peer Aid: Cash in the News by GiveDirectly - 8 Links about GiveDirectly, cash transfer and basic income.

The Value Of Money Going To Different Groups by The Center for Effective Altruism - "It is well known that an extra dollar is worth less when you have more money. This paper describes the way economists typically model that effect, using that to compare the effectiveness of different interventions. It takes remittances as a particular case study."

Politics and Economics:

Study Of The Week Better And Worse Ways To Attack Entrance Exams by Freddie deBoer - Freddie's description of four forms of "test validity". The SAT and ACT are predictive of college grades, one should criticize them from other angles. Freddie briefly gives his socialist critique.

How To Destroy Civilization by Zvi Moshowitz - A parable about the game "Advanced Civilization". The difficulties of building a coalition to lock out bad actor. Donald Trump. [Extremely Partisan]

Trust Assimilation by Bryan Caplan - Data on how much immigrants and their children trust other people. How predictive is the trust level of their ancestral country. Caplan reviews papers and crunches the numbers himself.

There Are Bots, Look Around by Renee DiResta (ribbonfarm) - High frequency trading disrupted finance. Now algorithms and bots are disrupting the marketplace of ideas. What can finance's past teach us about politics' future?

The Behavioral Economics of Paperwork by Bryan Caplan - Vast Numbers of students miss financial aid because they don't fill out paperwork. Caplan explores the economic implications of the fact that "Humans hate filling out paperwork. As a result, objectively small paperwork costs plausibly have huge behavioral response".

The Nimby Challenge by Noah Smith - Smith Argues makes an economic counterargument to the claims that building more housing wouldn't lower prices. Noah includes 6 lessons for engaging with NIMBYs.

Study Of The Week What Actually Helps Poor Students: Human Beings by Freddie deBoer - Personal feedback, tutoring and small group instruction had the largest positive effect. Includes Freddie's explanation of meta-analysis.

Vast Empirical Literature by Marginal REVOLUTION - Tyler's 10 thoughts on approaching fields with large literatures. He is critical of Noah's "two paper rule" and recommends alot of reading.

Impact Housing Price Restrictions by Marginal REVOLUTION - Link to a job market paper on the economic effects of housing regulation.

Me On Anarcho Capitalism by Bryan Caplan - Bryan is interviewed on the Rubin Report about Ancap.

Campbells Law And The Inevitability Of School Fraud by Freddie deBoer - Rampant Grade Inflation. Lowered standards. Campbell's law says that once you base policy on a metric that metric will always start being gamed

Nimbys Economic Theories: Sorry Not Sorry by Phil (Gelman's Blog) - Gelman got a huge amount of criticism on his post on whether building more housing will lower prices in the Bay. He responds to some of the criticism here. Long for Gelman.

Links 8 by Artir (Nintil) - Link Post. Physics, Technology, Philosophy, Economics, Psychology and Misc.

Arguing About How The World Should Burn by Sonya Mann ribbonfarm - Two different ways to decide who to exclude. One focuses on process the other on content. Scott Alexander and Nate Soares are quoted. Heavily [Culture War].

Seeing Like A State by Bayesian Investor - A quick review of "Seeing like a state".

Whats Up With Minimum Wage by Sarah Constantin (Otium) - A quick review of the literature on the minimum wage. Some possible explanations for why raising it not reduce unemployment.


Entirely Too Many Pieces Of Unsolicited Advice To Young Writer Types by Feddie deBoer - Advice about not working for free, getting paid, interacting with editors, why 'Strunk and White' is awful, and taking writing seriously.

Conversations On Consciousness by H i v e w i r e d - The author is a plural system. Their hope is to introduce plurality by doing the following: "First, we’re each going to describe our own personal experiences, from our own perspectives, and then we’re going to discuss where we might find ourselves within the larger narrative regarding consciousness."

Notes On Debugging Clojure Code by Eli Bendersky - Dealing with Clojure's cryptic exceptions, Finding which form an exception comes from, Trails and Logging, Deeper tracing inside cond forms

How to Think Scientifically About Scientists’ Proposals for Fixing Science by Andrew Gelman - Gelman asks how to scientifically evaluate proposals to fix science. He considers educational, statistical, research practice and institutional reforms. Excerpts from an article Gelman wrote, the full paper is linked.

Call for Volunteers who Want to Exercize by Aceso Under Glass - Author is looking for volunteers who want to treat their anxiety or mood disorder with exercise.

Learning Deep Learning the Easy Way with Keras (lesswrong) - Articles showing the power of neural networks. Discussion of ML frameworks. Resources for learning.

Unsong of Unsongs by Scott Aaronson - Aaronson went to the Unsong wrap party. A quick review of Unsong. Aaronson talks about how Scott Alexander defended him with untitled.

2016 Spending by Mr. Money Mustache - Full details of last year's budget. Spending broken down by category.


And Another Physics Problem by protokol2020 - Two Planets. Which has a higher average surface temperature.

A mysterious jogger by Jacob Falkovich (Put A Number On It!) - A mysterious jogger. Very short fiction.


Persuasion And Control by Waking Up with Sam Harris - "surveillance capitalism, the Trump campaign's use of Facebook, AI-enabled marketing, the health of the press, Wikileaks, ransomware attacks, and other topics."

Raj Chetty: Inequality, Mobility and the American Dream by Conversations with Tyler - "As far as I can tell, this is the only coverage of Chetty that covers his entire life and career, including his upbringing, his early life, and the evolution of his career, not to mention his taste in music"

Is Trump's incompetence saving us from his illiberalism? by The Ezra Klein Show - Political Scientist Yascha Mounk. "What Mounk found is that the consensus we thought existed on behalf of democracy and democratic norms is weakening."

The Moral Complexity Of Genetics by Waking Up with Sam Harris - "Sam talks with Siddhartha Mukherjee about the human desire to understand and manipulate heredity, the genius of Gregor Mendel, the ethics of altering our genes, the future of genetic medicine, patent issues in genetic research, controversies about race and intelligence, and other topics."

Ester Perel by The Tim Ferriss - The Relationship Episode: Sex, Love, Polyamory, Marriage, and More

Lane Pritchett by Econtalk - Growth, and Experiments

Meta Learning by Tim Ferriss - Education, accelerated learning, and my mentors. Conversation with Charles Best the founder and CEO of

Bryan Stevenson On Why The Opposite Of Poverty Isn't Wealth by The Ezra Klein Show - Founder and executive director of the Equal Justice Initiative. Justice for the wrongly convicted on Death Row.

The dark arts: Examples from the Harris-Adams conversation

15 Stabilizer 20 July 2017 11:42PM

Recently, James_Miller posted a conversation between Sam Harris and Scott Adams about Donald Trump. James_Miller titled it "a model rationalist disagreement". While I agree that the tone in which the conversation was conducted was helpful, I think Scott Adams is a top practitioner of the Dark Arts. Indeed, he often prides himself on his persuasion ability. To me, he is very far from a model for a rationalist, and he is the kind of figure we rationalists should know how to fight against.


Here are some techniques that Adams uses:


  1. Changing the subject: (a) Harris says Trump is unethical and cites the example of Trump gate-crashing a charity event to falsely get credit for himself. Adams responds by saying that others are equally bad—that all politicians do morally dubious things. When Harris points out that Obama would never do such a thing, Adams says Trump is a very public figure and hence people have lots of dirt on him. (b) When Harris points out that almost all climate scientists agree that climate change is happening and that it is wrong for Trump to have called climate change a hoax, Adams changes the subject to how it is unclear what economic policies one ought to pursue if climate change is true.
  2. Motte-and-bailey: When Harris points out that the Trump University scandal and Trump's response to it means Trump is unethical, Adams says that Trump was not responsible for the university because it was only a licensing deal. Then Harris points out that Trump is unethical because he shortchanged his contractors. Adams says that that’s what happens with big construction projects. Harris tries to argue that it’s the entirety of Trump’s behavior that makes it clear that he is unethical—i.e., Trump University, his non-payment to contractors, his charity gate-crashing, and so on. At this points Adams says we ought to stop expecting ethical behavior from our Presidents. This is a classic motte-and-bailey defense. Try to defend an indefensible position (the bailey) for a while, but then once it becomes untenable to defend it, then go to the motte (something much more defensible).
  3. Euphemisation: (a) When Harris tells Adams that Trump lies constantly and has a dangerous disregard for the truth, Adams says, I agree that Trump doesn’t pass fact checks. Indeed, throughout the conversation Adams never refers to Trump as lying or as making false statements. Instead, Adams always says, Trump “doesn’t pass the fact checks”. This move essentially makes it sound as if there’s some organization whose arbitrary and biased standards are what Trump doesn’t pass and so downplays the much more important fact that Trump lies. (b) When Harris call Trump's actions morally wrong, Adams makes it seem as if he is agreeing with Harris but then rephrases it as: “he does things that you or I may not do in the same situation”. Indeed, that's Adams's constant euphemism for a morally wrong action. This is a very different statement compared to saying that what Trump did was wrong, and makes it seem as if Trump is just a normal person doing what normal people do. 
  4. Diagnosis: Rather than debate the substance of Harris’s claims, Adams will often embark on a diagnosis of Harris’s beliefs or of someone else who has that belief. For example, when Harris says that Trump is not persuasive and does not seem to have any coherent views, Adams says that that's Harris's "tell" and that Harris is "triggered" by Trump's speeches. Adams constantly diagnoses Trump critics as seeing a different movie, or as being hypnotized by the mainstream media. By doing this, he moves away from the substance of the criticisms.
  5. Excusing: (a) When Harris says that it is wrong to not condemn, and wrong to support, the intervention of Russia in America’s election, Adams says that the US would extract revenge via its intelligence agencies and we would never know about it. He provides no evidence for the claim that Trump is indeed extracting revenge via the CIA. He also says America interferes in other elections too. (b) When Harris says that Trump degraded democratic institutions by promising to lock up his political opponent after the election, Adams says that was just a joke. (c) When Harris says Trump is using the office of the President for personal gain, Adams tries to spin the narrative as Trump trying to give as much as possible late in his life for his country. 
  6. Cherry-picking evidence: (a) When Harris points out that seventeen different intelligence agencies agreed that Russia’s government interfered in the US elections, Adams says that the intelligence agencies have been known to be wrong before. (b) When Harris points out that almost all climate scientists agree on climate change, Adams points to some point in the 1970s where (he claims) climate scientists got something wrong, and therefore we should be skeptical about the claims of climate scientists.

Overall, I think what Adams is doing is wrong. He is an ethical and epistemological relativist: he does not seem to believe in truth or in morality. At the very least, he does not care about what is true and false and what is right and wrong. He exploits his relativism to push his agenda, which is blindingly clear: support Trump.


(Note: I wanted to work on this essay more carefully, and find out all the different ways in which Adams subverts the truth and sound reasoning. I also wanted to cite more clearly the problematic passages from the conversations. But I don't have the time. So I relied on memory and highlighted the Dark Arts moves that struck me immediately. So please, contribute in the comments with your own observations about the Dark Arts involved here.)

Thought experiment: coarse-grained VR utopia

15 cousin_it 14 June 2017 08:03AM

I think I've come up with a fun thought experiment about friendly AI. It's pretty obvious in retrospect, but I haven't seen it posted before. 

When thinking about what friendly AI should do, one big source of difficulty is that the inputs are supposed to be human intuitions, based on our coarse-grained and confused world models. While the AI's actions are supposed to be fine-grained actions based on the true nature of the universe, which can turn out very weird. That leads to a messy problem of translating preferences from one domain to another, which crops up everywhere in FAI thinking, Wei's comment and Eliezer's writeup are good places to start.

What I just realized is that you can handwave the problem away, by imagining a universe whose true nature agrees with human intuitions by fiat. Think of it as a coarse-grained virtual reality where everything is built from polygons and textures instead of atoms, and all interactions between objects are explicitly coded. It would contain player avatars, controlled by ordinary human brains sitting outside the simulation (so the simulation doesn't even need to support thought).

The FAI-relevant question is: How hard is it to describe a coarse-grained VR utopia that you would agree to live in?

If describing such a utopia is feasible at all, it involves thinking about only human-scale experiences, not physics or tech. So in theory we could hand it off to human philosophers or some other human-based procedure, thus dealing with "complexity of value" without much risk. Then we could launch a powerful AI aimed at rebuilding reality to match it (more concretely, making the world's conscious experiences match a specific coarse-grained VR utopia, without any extra hidden suffering). That's still a very hard task, because it requires solving decision theory and the problem of consciousness, but it seems more manageable than solving friendliness completely. The resulting world would be suboptimal in many ways, e.g. it wouldn't have much room for science or self-modification, but it might be enough to avert AI disaster (!)

I'm not proposing this as a plan for FAI, because we can probably come up with something better. But what do you think of it as a thought experiment? Is it a useful way to split up the problem, separating the complexity of human values from the complexity of non-human nature?

Mode Collapse and the Norm One Principle

15 tristanm 05 June 2017 09:30PM

[Epistemic status: I assign a 70% chance that this model proves to be useful, 30% chance it describes things we are already trying to do to a large degree, and won't cause us to update much.] 

I'm going to talk about something that's a little weird, because it uses some results from some very recent ML theory to make a metaphor about something seemingly entirely unrelated - norms surrounding discourse. 

I'm also going to reach some conclusions that surprised me when I finally obtained them, because it caused me to update on a few things that I had previously been fairly confident about. This argument basically concludes that we should adopt fairly strict speech norms, and that there could be great benefit to moderating our discourse well. 

I argue that in fact, discourse can be considered an optimization process and can be thought of in the same way that we think of optimizing a large function. As I will argue, thinking of it in this way will allow us to make a very specific set of norms that are easy to think about and easy to enforce. It is partly a proposal for how to solve the problem of dealing with speech that is considered hostile, low-quality, or otherwise harmful. But most importantly, it is a proposal for how to ensure that the discussion always moves in the right direction: Towards better solutions and more accurate models. 

It will also help us avoid something I'm referring to as "mode collapse" (where new ideas generated are non-diverse and are typically characterized by adding more and more details to ideas that have already been tested extensively). It's also highly related to the concepts discussed in the Death Spirals and the Cult Attractor portion of the Sequences. Ideally, we'd like to be able to make sure that we're exploring as much of the hypothesis space as possible, and there's good reason to believe we're probably not doing this very well.  

The challenge: Making sure we're searching for the global optimum in model-space sometimes requires reaching out blindly into the frontiers, the not well-explored regions, which runs the risk of ending up somewhere very low-quality or dangerous. There are also sometimes large gaps between very different regions of model-space where the quality of the model is very low in-between, but very high on each side of the gap. This requires traversing through potentially dangerous territory and being able to survive the whole way through.

(I'll be using terms like "models" and "hypotheses" quite often, and I hope this isn't confusing. I am using them very broadly, to refer to both theoretical understandings of phenomenon and blueprints for practical implementations of ideas). 

We desire to have a set of principles which allows us to do this safely - to think about models of the world that are new and untested, solutions for solving problems that have never been done in a similar way - and they should ensure that, eventually, we can reach the global optimum. 

Before we derive that set of principles, I am going to introduce a topic of interest from the field of Machine Learning. This topic will serve as the main analogy for the rest of this piece, and serve as a model for how the dynamics of discourse should work in the ideal case. 

I. The Analogy: Generative Adversarial Networks

For those of you who are not familiar with the recent developments in deep-learning, Generative Adversarial Networks (GANs)[intro pdf here] are a new type of generative model class that are ideal for producing high-quality samples from very high-dimensional, complex distributions. They have caused great buzz and hype in the deep-learning community due to how impressive some of the samples they produce are, and how efficient they are at generation.

Put simply, a generator model and a critic (sometimes called a discriminator) model perform a two-player game where the critic is trained to distinguish between samples produced by the generator and the "true" samples taken from the data distribution. In turn, the generator is trained to maximize the critic's loss function. Both models are usually parametrized by deep neural networks and can be trained by taking turns running a gradient descent step on each. The Nash equilibrium of this game is when the generator's distribution matches that of the data distribution perfectly. This is never really borne out in practice, but sometimes it gets so close that we don't mind. 

GANs have one principal failure mode, which is often thought to be due to the instability of the system, which is often called "mode collapse" (a term I'm going to appropriate to refer to a much broader concept). It was often believed that, if a careful balance between the generator and critic could not be maintained, one would eventually overpower the other - leading the critic to provide either useless or overly harsh information to the generator. Useless information will cause the generator to update very slowly or not at all, and overly harsh information will lead the samples to "collapse" to a small region of the data space that are the easiest targets for the generator to hit.  

This problem was essentially solved earlier this year due to a series of papers that propose modifications to the loss functions that GANs use, and, most crucially, add another term to the critic's loss which stabilizes the gradient (with respect to the inputs) to have a norm close to one. It was recognized that we actually desire an extremely powerful critic so that the generator can make the best updates it possibly can, but the updates themselves can't go beyond what the generator is capable of handling. With these changes to the GAN formulation, it became possible to use crazy critic networks such as ultra-deep ResNets and train them as much as desired before updating the generator network.  

The principle behind their operation is rather simple to describe, but unfortunately, it is much more difficult to explain why they work so well. However, I believe that as long as we know how to make one, and know specific implementation details that improve their stability, then I believe their principles can be applied more broadly to achieve success in a wide variety of regimes. 

II. GANs as a Model of Discourse

In order to use GANs as a tool for conceptual understanding of discourse, I propose to model of the dynamics of debate as a collection of hypothesis-generators and hypothesis-critics. This could be likened to the structure of academia - researchers publish papers, they go through peer-review, the work is iterated on and improved - and over time this process converges to more and more accurate models of reality (or so we hope). Most individuals within this process play both roles, but in theory this process would still work even if they didn't. For example, Isaac Newton was a superb hypothesis generator, but he also had some wacky ideas that most of us would consider to be obviously absurd. Nevertheless, calculus and Newtonian physics became a part of our accepted scientific knowledge, and alchemy didn't. The system adopted and iterated on his good ideas while throwing away the bad. 

Our community should be capable of something similar, while doing it more efficiently and not requiring the massive infrastructure of academia. 

A hypothesis-generator is not something that just randomly pulls out a model from model-space. It proposes things that are close modifications of things it already holds to be likely within its model (though I expect this point to be debatable). Humans are both hypothesis-generators and hypothesis-critics. And as I will argue, that distinction is not quite as sharply defined as one would think. 

I think there has always been an underlying assumption within the theory of intelligence that creativity and recognition / distinction are fundamentally different. In other words, one can easily understand Mozart to be a great composer, but it is much more difficult to be a Mozart. Naturally this belief entered it's way into the field of Artificial Intelligence too, and became somewhat of a dogma. Computers might be able to play Chess, they might be able to play Go, but they aren't doing anything fundamentally intelligent. They lack the creative spark, they work on pure brute-force calculation only, with maybe some heuristics and tricks that their human creators bestowed upon them.  

GANs seem to defy this principle. Trained on a dataset of photographs of human faces, a GAN generator learns to produce near-photo-realistic images that nonetheless do not fully match any the faces the critic network saw (one of the reasons why CelebA was such a good choice to test these on), and are therefore in some sense producing things which are genuinely original. It may have once been thought that there was a fundamental distinction between creation and critique, but perhaps that's not really the case. GANs were a surprising discovery, because they showed that it was possible to make impressive "creations" by starting from random nonsense and slowly tweaking it in the direction of "good" until it eventually got there (well okay, that's basically true for the whole of optimization, but it was thought to be especially difficult for generative models).

What does this mean? Could someone become a "Mozart" by beginning a musical composition from random noise and slowly tweaking it until it became a masterpiece?

The above seems to imply "yes, perhaps." However, this is highly contingent on the quality of the "tweaking." It seems possible only as long as the directions to update in are very high quality. What if they aren't very high quality? What if they point nowhere, or in very bad directions?

I think the default distribution of discourse is that it is characterized by a large number of these directionless, low quality contributions. And that it's likely that this is one of the main factors behind mode collapse. This is related to what has been noted before: Too much intolerance for imperfect ideas (or ideas outside of established dogma) in a community prevent useful tasks from being accomplished, and progress from being made. Academia does not seem immune to this problem. Where low-quality or hostile discussion is tolerated is where this risk is greatest.  

Fortunately, making sure we get good "tweaks" seems to be the easy part. Critique is in high abundance. Our community is apparently very good at it. We also don't need to worry much about the ratio of hypothesis-generators to hypothesis-critics, as long as we can establish good principles that allow us to follow GANs as closely as possible. The nice feature of the GAN formulation is that you are allowed to make the critic as powerful as you want. In fact, the critic should be more powerful than the generator (If the generator is too powerful, it just goes directly to the argmax of the critic). 

(In addition, any collection of generators is a generator, and any collection of critics is a critic. So this formulation can be applied to the community setting).

III. The Norm One Principle

So the question then becomes, how do we take an algorithm governing a game between models much simpler than a human, and use the same tweaks which consist of nothing more than a few very simple equations? 

Here what I devise is a strategy for taking the concept of the norm of the critic gradient being as close to one as possible, and using that as a heuristic for how to structure appropriate discourse. 

(This is where my argument gets more speculative and I expect to update this a lot, and where I welcome the most criticism).

What I propose is that we begin modeling the concept of "criticism" based on how useful it is to the idea-generator receiving the criticism. Under this model, I think we should start breaking down criticism into two fundamental attributes:

  1. Directionality - does the criticism contain highly useful information, such that the "generator" knows how to update their model / hypothesis / proposal?
  2. Magnitude - Is the criticism too harsh, does it point to something completely unlike the original proposal, or otherwise require changes that aren't feasible for the generator to make?

My claim is that any contribution to a discussion should satisfy the "Norm One Principle." In other words, it should have a well-defined direction, and the quantity of change should be feasible to implement.

If a critique can satisfy our requirements for both directionality and magnitude, then it serves a useful purpose. The inverse claim to this is that if we can't follow these requirements, we risk falling into mode collapse, and the ideas commonly proposed are almost indistinguishable from the ones which preceded them, and ideas which deviate too far from the norm are harshly condemned and suppressed. 

I think it's natural to question whether or not restricting criticism to follow certain principles is a form of speech suppression that prevents useful ideas from being considered. But the pattern I'm proposing doesn't restrict the "generation" process, the creative aspect which produces new hypotheses. It doesn't restrict the topics that can be discussed. It only restricts the criticism of those hypotheses, such that they are maximally useful to the source of the hypothesis. 

One of the primary fears behind having too much criticism is that it discourages people from contributing because they want to avoid the negative feedback. But under the Norm One Principle, I think it is useful to distinguish between disagreement and criticism. I think if we're following these norms properly, we won't need to consider criticism to be a negative reward. In fact, criticism can be positive. Agreement could be considered "criticism in the same direction you are moving in." Disagreement would be the opposite. And these norms also eliminate the kind of feedback that tends to be the most discouraging. 

For example, some things which violate "Norm One":

  • Ad hominem attacks (typically directionless). 
  • Affective Death Spirals (unlimited praise or denunciation is usually directionless, and usually very high magnitude). 
  • Signs that cause aversion (things I "don't like", that trigger my System 1 alarms, which probably violates both directionality and magnitude). 
  • Lengthy lists of changes to make (norm greater than 1, ideally we want to try to focus on small sets of changes that have the highest priority). 
  • Repetition of points that have already been made (norm greater than one). 

One of my strongest hopes is that whomever is playing the part of the "generator" is able to compile the list of critiques easily and use them to update somewhere close to the optimal direction. This would be difficult if the sum of all critiques is either directionless (many critics point in opposite or near-opposite directions) or very high-magnitude (Critics simply say to get as far away from here as possible). 

But let's suppose that each individual criticism satisfies the Norm One principle. We will also assume that the generator is weighing each critique by their respect for whoever produced it, which I think is highly likely. Then the generator should be able to move in a direction unless the sum of the directions completely cancel out. It is unlikely for this to happen - unless there is very strong epistemic disagreement in the community over some fundamental assumptions (in which case the conversation should probably move over to that). 

In addition, it also becomes less likely for the directions to cancel out as the number of inputs increases. Thus, it seems that proposals for new models should be presented to a wide audience, and we should avoid the temptation to keep our proposals hidden to all except for a small set of people we trust.

So I think that in general, this proposed structure should tend to increase the amount of collective trust we have in the community, and that it favors transparency and favors diversity of viewpoints. 

But what of the possible failure modes of this plan? 

This model should fail if the specific details of its implementation either remove too much discussion, or fail to deal with individuals who refuse to follow the norms and refuse to update. Any implementation should allow room for anyone to update. Someone who posts an extremely hostile, directionless comment should be allowed chances to modify their contribution. The only scenario in which the "banhammer" becomes appropriate is when this model fails to apply: The cardinal sin of rationality, the refusal to update. 

IV. Building the Ideal "Generator"

As a final point, I'll note that the above assumes that generators will be able to update their models incrementally. The easy part, as I mentioned, was obtaining the updates, the hard part is accumulating them. This seems difficult with the infrastructure we have in place. What we do have is a good system for posting proposals and receiving feedback (The blog post / comment thread set-up), but this assumes that each "generator" is keeping track of their models by themselves and has to be fully aware of the status of other models on their own. There is no centralized "mixture model" anywhere that contains the full set of models weighted by how much probability they are given by the community. Currently, we do not have a good solution for this problem. 

However, it seems that the first conception of Arbital was centered around finding a solution to this kind of problem:

Arbital has bigger ambitions than even that. We all dream of a world that eliminates the duplication of effort in online argument - a world where, the same way that Wikipedia centralized the recording of definite facts, an argument only needs to happen once, instead of being reduplicated all over the Internet; with all the branches of the argument neatly recorded in the same place, along with some indication of who believes what. A world where 'just check Arbital' had the same status for determining the current state of debates, as 'just check Wikipedia' now has when somebody starts arguing about the population of Melbourne. There's entirely new big subproblems and solutions, not present at all in the current Arbital, that we'd need to tackle that considerably more difficult problem. But to solve 'explaining things' is something of a first step. If you have a single URL that you can point anyone to for 'explaining Bayes', and if you can dispatch people to different pages depending on how much math they know, you're starting to solve some of the key subproblems in removing the redundancy in online arguments.

If my proposed model is accurate, then it suggests that the problem Arbital aims to solve is in fact quite crucial to solve, and that the developers of Arbital should consider working through each obstacle they face without pivoting from this original goal. I feel confident enough that this goal should be high priority that I'd be willing to support its development in whatever way is deemed most helpful and is feasible for me (I am not an investor, but I am a programmer and would also be capable of making small donations, or contributing material). 

The only thing that this model would require for Arbital to do would be to make it as open as possible to contribute, and then perform heavy moderation or filtering of contributed content (but importantly not the other way around, where it is closed to small group of trusted people).

Currently, the incremental changes that would have to be made to LessWrong and related sites like SSC would simply be increased moderation of comment quality. Otherwise, any further progress on the problem would require overcoming much more serious obstacles requiring significant re-design and architecture changes. 

Everything I've written above is also subject to the model I've just outlined, and therefore I expect to make incremental updates as feedback to this post accrues.

My initial prediction for feedback to this post is that the ideas might be considered helpful and offer a useful perspective or a good starting point, but that there are probably many details that I have missed that would be useful to discuss, or points that were not quite well-argued or well thought-out. I will look out for these things in the comments.   

[Link] Game Theory & The Golden Rule (From Reddit)

14 Brillyant 28 July 2017 01:54PM

Becoming stronger together

14 b4yes 11 July 2017 01:00PM

I want people to go forth, but also to return.  Or maybe even to go forth and stay simultaneously, because this is the Internet and we can get away with that sort of thing; I've learned some interesting things on Less Wrong, lately, and if continuing motivation over years is any sort of problem, talking to others (or even seeing that others are also trying) does often help.

But at any rate, if I have affected you at all, then I hope you will go forth and confront challenges, and achieve somewhere beyond your armchair, and create new Art; and then, remembering whence you came, radio back to tell others what you learned.

Eliezer Yudkowsky, Rationality: From AI to Zombies

If you want to go fast, go alone. If you want to go far, go together.

African proverb (possibly just made up)

About a year ago, a secret rationalist group was founded. This is a report of what the group did during that year.

The Purpose

“Rationality, once seen, cannot be unseen,” are words that resonate with all of us. Having glimpsed the general shape of the thing, we feel like we no longer have a choice. I mean, of course we still have an option to think and act in stupid ways, and we probably do it a lot more than we would be okay to admit! We just no longer have an option to do it knowingly without feeling stupid about it. We can stray from the way, but we cannot pretend anymore that it does not exist. And we strongly feel that more is possible, both in our private lives, and for the society in general.

Less Wrong is the website and the community that brought us together. Rationalist meetups are a great place to find smart, interesting, and nice people; awesome people to spend your time with. But feeling good was not enough for us; we also wanted to become stronger. We wanted to live awesome lives, not just to have an awesome afternoon once in a while. But many participants seemed to be there only to enjoy the debate. Or perhaps they were too busy doing important things in their lives. We wanted to achieve something together; not just as individual aspiring rationalists, but as a rationalist group. To make peer pressure a positive force in our lives; to overcome akrasia and become more productive, to provide each other feedback and to hold each other accountable, to support each other. To win, both individually and together.

The Group

We are not super secret really; some people may recognize us by reading this article. (If you are one of them, please keep it to yourself.) We just do not want to be unnecessarily public. We know who we are and what we do, and we are doing it to win at life; trying to impress random people online could easily become a distraction, a lost purpose. (This article, of course, is an exception.) This is not supposed to be about specific individuals, but an inspiration for you.

We started as a group of about ten members, but for various reasons some people soon stopped participating; seven members remained. We feel that the current number is probably optimal for our group dynamic (see Parkinson's law), and we are not recruiting new members. We have a rule “what happens in the group, stays in the group”, which allows us to be more open to each other. We seem to fit together quite well, personality-wise. We desire to protect the status quo, because it seems to work for us.

But we would be happy to see other groups like ours, and to cooperate with them. If you want to have a similar kind of experience, we suggest starting your own group. Being in contact with other rationalists, and holding each other accountable, seems to benefit people a lot. CFAR also tries to keep their alumni in regular contact after the rationality workshops, and some have reported this as a huge added value.

To paint a bit more specific picture of us, here is some summary data:

  • Our ages are between 20 and 40, mostly in the middle of the interval.
  • Most of us, but not all, are men.
  • Most of us, but not all, are childless.
  • All of us are of majority ethnicity.
  • Most of us speak the majority language as our first language.
  • All of us are atheists; most of us come from atheist families.
  • Most of us have middle-class family background.
  • Most of us are, or were at some moment, software developers.

I guess this is more or less what you could have expected, if you are already familiar with the rationalist community.

We share many core values, but have some different perspectives, which adds value and confronts groupthink. We have entrepreneurs, employees, students, and unemployed bums; the ratio changes quite often. It is probably the combination of all of us having a good sense of epistemology, but different upbringing, education and professions, that makes supporting each other and giving advice more effective (i.e. beyond the usual benefits of the outside view); there have been plenty of situations which were trivial for one, but not for the other.

Some of us knew each other for years before starting the group, even before the local Less Wrong meetups. Some of us met the others at the meetups. And finally, some of us talked to some other members for the first time after joining the group. It is surprising how well we fit, considering that we didn’t apply any membership filter (although we were prepared to); people probably filtered themselves by their own interest, or a lack thereof, to join this kind of a group, specifically with the productivity and accountability requirements.

We live in different cities. About once in a month we meet in person; typically before or after the local Less Wrong meetup. We spend a weekend together. We walk around the city and debate random stuff in the evening. In the morning, we have a “round table” where each of us provides a summary of what they did during the previous month, and what they are planning to do during the following month; about 20 minutes per person. That takes a lot of time, and you have to be careful not to go off-topic too often.

In between meetups, we have a Slack team that we use daily. Various channels for different topics; the most important one is a “daily log”, where members can write briefly what they did during the day, and optionally what they are planning to do. In addition to providing extra visibility and accountability, it helps us feel like we are together, despite the geographical distances.

Besides mutual accountability, we are also fans of various forms of self-tracking. We share tips about tools and techniques, and show each other our data. Journaling, time tracking, exercise logging, step counting, finance tracking...

Even before starting the group, most of us were interested in various productivity systems: Getting Things Done, PJ Eby; one of us even wrote and sold their own productivity software.

We do not share a specific plan or goal, besides “winning” in general. Everyone follows their own plan. Everything is voluntary; there are no obligations nor punishments. Still, some convergent goals have emerged.

Also, good habits seem to be contagious, at least in our group. If a single person was doing some useful thing consistently, eventually the majority of the group seems to pick it up, whether it is related to productivity, exercise, diet, or finance.


All of us exercise regularly. Now it seems like obviously the right thing to do. Exercise improves your health and stamina, including mental stamina. For example, the best chess players exercise a lot, because it helps them stay focused and keep thinking for a long time. Exercise increases your expected lifespan, which should be especially important for transhumanists, because it increases your chances to survive until the Singularity. Exercise also makes you more attractive, creating a halo effect that brings many other benefits.

If you don’t consider these benefits worth at least 2 hours of your time a week, we find it difficult to consider you a rational person who takes their ideas seriously. Yes, even if you are busy doing important things; the physical and mental stamina gained from exercising is a multiplier to whatever you are doing in the rest of your time.

Most of us lift weights (see e.g. StrongLifts 5×5, Alan Thrall); some of us even have a power rack and/or treadmill desk at home. Others exercise using their body weight (see Convict Conditioning). Exercising at home saves time, and in long term also money. Muscle mass correlates with longevity, in addition to the effect of exercise itself; and having more muscle allows you to eat more food. Speaking of which...


Most of us are, mostly or completely, vegetarian or vegan. Ignoring the ethical aspects and focusing only on health benefits, there is a lot of nutrition research summarized in a book How Not to Die and a website The short version is that whole-food vegan diet seems to work best, but you really should look into details. (Not all vegan food is automatically healthy; there is also vegan junk food. It is important to eat a lot of unprocessed vegetables, fruit, nuts, flax seeds, broccoli, beans. Read the book, seriously. Or download the Daily Dozen app.) We often share tasty recipes when we meet.

We also helped each other research food supplements, and actually find the best and cheapest sources. Most of us take extra B12 to supplement the vegan diet, creatine monohydrate, vitamin D3, and some of us also use Omega3, broccoli sprouts, and a couple of other things that are generally aimed at health and longevity.


We strategize and brainstorm career decisions or just debug office politics. Most of us are software developers. This year, one member spent nine months learning how to program (using Codecademy, Codewars, and freeCodeCamp at the beginning; reading tutorials and documentation later); as a result their income more than doubled, and they got a job they can do fully remotely.

Recently we started researching cryptocurrencies and investing in them. Some of us started doing P2P lending.

Personal life

Many of us are polyamorous. We openly discuss sex and body image issues in the group. We generally feel comfortable sharing this information with each other; women say they do not feel the typical chilling effects.


Different members report different benefits from their membership in the group. Some quotes:

“During the first half of the year, my life was more or less the same. I was already very productive before the group, so I kept the same habits, but benefited from sharing research. Recently, my life changed more noticeably. I started training myself to think of more high-leverage moves (inspired by a talk on self-hypnosis). This changed my asset allocation, and my short-term career plans. I realize more and more that I am very much monkey see, monkey do.”

“Before stumbling over the local Less Wrong meetup, I had been longing (and looking) for people who shared, or even just understood, my interest and enthusiasm for global, long-term, and meta thinking (what I now know to be epistemic rationality). After the initial thrill of the discovery had worn off however, I soon felt another type of dissonance creeping up on me: "Wait, didn't we agree that this was ultimately about winning? Where is the second, instrumental half of rationality, that was supposedly part of the package?" Well, it turned out that the solution to erasing this lingering dissatisfaction was to be found in yet a smaller subgroup.

So, like receiving a signal free of interference for the first time, I finally feel like I'm in a "place" where I can truly belong, i.e. a tribe, or at least a precursor to one, because I believe that things hold the potential to be way more awesome still, and that just time alone may already be enough to take us there.

On a practical level, the speed of adoption of healthy habits is truly remarkable. I've always been able to generally stick to any goals and commitments I've settled on, however the process of convergence is just so much faster and easier when you can rely on the judgment of other epistemically trustworthy people. Going at full speed is orders of magnitudes easier when multiple people illuminate the path (i.e. figure out what is truly worth it), while simultaneously sharing the burdens (of research, efficient implementation, trial-and-error, etc.)”

“Now I'm on a whole-food vegan diet and I exercise 2 times a week, and I also improved in introspection and solving my life problems. But most importantly, the group provides me companionship and emotional support; for example, starting a new career is a lot easier in the presence of a group where reinventing yourself is the norm.”

“It usually takes grit and willpower to change if you do it alone; on the other hand, I think it's fairly effortless if you're simply aligning your behavior with a preexisting strong group norm. I used to eat garbage, smoke weed, and have no direction in life. Now I lift weights, eat ~healthy, and I learned programming well enough to land a great job.

The group provides existential mooring; it is a homebase out of which I can explore life. I don't think I'm completely un-lost, but instead of being alone in the middle of a jungle, I'm at a friendly village in the middle of a jungle.”

“I was already weightlifting and eating vegan, but got motivated to get more into raw and whole foods. I get confronted more with math, programming and finance, and can broaden my horizon. Sharing daily tasks in Slack helps me to reflect about my priorities. I already could discuss many current career and personal challenges with the whole group or individuals.”

“I started exercising regularly, and despite remaining an omnivore I eat much more fresh vegetables now than before. People keep telling me that my body shape improved a lot during this year. Other habits did not stick (yet).”

“Finding a tribe of sane people in an insane world was a big deal for me, now I feel more self-assured and less alone. Our tribe has helped me to improve my habits—some more than others (for example, it has inspired me to buy a power-rack for my living room and start weightlifting daily, instead of going to the gym). The friendly bragging we do among our group is our way of celebrating success and inspires me to keep going and growing.”


Despite having met each other thanks to Less Wrong, most of us do not read it anymore, because our impression is that “Less Wrong is dead”. We do read Slate Star Codex.

From other rationalist blogs, we really liked the article about Ra, and we discussed it a lot.

The proposal of a Dragon Army evoked mixed reactions. On one hand, we approve of rationalists living closer to each other, and we want to encourage fellow rationalists to try it. On the other hand, we don’t like the idea of living in a command hierarchy; we are adults, and we all have our own projects. Our preferred model would be living close to each other; optimally in the same apartment building with some shared communal space, but also with a completely self-contained unit for each of us. So far our shared living happened mostly by chance, but it always worked out very well.

Jordan Peterson and his Self-Authoring Suite is very popular with about half of the group.

What next?

Well, we are obviously going to continue doing what we are doing now, hopefully even better than before, because it works for us.

You, dear reader, if you feel serious about becoming stronger and winning at life, but are not yet a member of a productive rationalist group, are encouraged to join one or start one. Geographical distances are annoying, but Slack helps you overcome the intervals between meetups. Talking to other rationalists can be a lot of fun, but accountability can make the difference between productivity and mere talking. Remember: “If this is your first night at fight club, you have to fight!”

Even if it’s seemingly small things, such as doing an exercise or adding some fiber to your diet; these things, accumulated over time, can increase your quality of life a lot. The most important habit is the meta-habit of creating and maintaining good habits. And it is always easier when your tribe is doing the same thing.

Any questions? It may take some time for our hive mind to generate an answer, and in case of too many or too complex questions we may have to prioritize. Don’t feel shy, though. We care about helping others.


(This account was created for the purpose of making this post, and after a week or two it will stop being used. It may be resurrected after another year, or maybe not. Please do not send private messages; they will most likely be ignored.)

In praise of fake frameworks

14 Valentine 11 July 2017 02:12AM

Related to: Bucket errors, Categorizing Has Consequences, Fallacies of Compression

Followup to: Gears in Understanding

I use a lot of fake frameworks — that is, ways of seeing the world that are probably or obviously wrong in some important way.

I think this is an important skill. There are obvious pitfalls, but I think the advantages are more than worth it. In fact, I think the "pitfalls" can even sometimes be epistemically useful.

Here I want to share why. This is for two reasons:

  • I think fake framework use is a wonderful skill. I want it represented more in rationality in practice. Or, I want to know where I'm missing something, and Less Wrong is a great place for that.

  • I'm building toward something. This is actually a continuation of Gears in Understanding, although I imagine it won't be at all clear here how. I need a suite of tools in order to describe something. Talking about fake frameworks is a good way to demo tool #2.

With that, let's get started.

continue reading »

Models of human relationships - tools to understand people

13 Elo 29 July 2017 03:31AM

This post will not teach you the models here.  This post is a summary of the models that I carry in my head.  I have written most of the descriptions without looking them up (See Feynman notebook method).  If you have read a book on every one of these points they will make sense, as if you were shaking hands with an old acquaintance.  If you are seeing them for the first time, they won't make very much sense or they will feel like a surface trivial truth.

I can't make you read all the books but maybe I can offer you that the answer to social problems is surprisingly simple.  After reading enough books you start to see the overlap and realise they often are trying to talk about the same thing.  (i.e. NVC + Gottman go together well).

In fact if you were several independent dragon hunters trying to model an invisible beast and all of various people's homemade sensors kept going "ping" at similar events you would probably start to agree you were chasing the same monster.  Models should start to agree when they are talking about the same thing.  The variety of models should make it easier for different minds to connect to different parts of the answer.

All models are wrong, some models are useful.  Try to look at where the models converge.  That's where I find the truth.

1. The book Crucial Confrontations - Kerry Patterson
(without explaining how) If you can navigate to a place of safety in a conversation you can say pretty much anything.  Which is not to say "here is how to be a jerk" but if you know something is going to come across negative you can first make sure to be in a positive/agreeable/supportive conversation before raising the hard thing.

In the middle of a yelling match is maybe not the best time to bring up something that has bugged you for years.  However a few sentences about growth mindset, supporting people being a better person and trying to help (and getting a feel that the person is ready to hear the thing) and you could tell anyone they are a lazy bum who needs to shape up or ship out.

The conversation needs to be safe.  For example - "I want to help you as a person and I know how hard it can be to get feedback from other people and I want to make you into a better person.  I have an idea for how you might like to improve.  Before I tell you I want to reassure you that even though this might come across abrasive I want to help you grow and be better in the future..."

(some people will be easier than others to navigate a safe conversation and that's where there are no hard and fast rules for how to do this.  Go with your gut)

The crux of this model is "have a model of the other person" [15]

2. The partner book "Difficult conversations"

There are 4 types of difficult conversations around communicating a decision:
a. Consultation (Bob asks Alice for ideas for the decision he is going to make on his own)
b. Collaboration (Bob and Alice make a decision together)
c. Declaration (Bob tells alice the decision he has made)
d. Delegation (Bob tells alice to make the decision)

As someone's boss you may sometimes have to pass on bad news in the form of a declaration.  It's up to you which conversation this is going to be but being clear about what conversation this is will be helpful to a person to understand their place in responding or interacting with you.  It becomes difficult where there is a misunderstanding about what is going on.

It's also important when you are on the receiving end to be on the same page about what conversation this is.  (you don't want to be negotiating in a collaborative manner when they are trying to give you a declaration of their decision, and the same when you are leading the conversation).

Among other details in the book.

3. Getting the 3rd story.

linking back to -
(from one of those books [1] or [2])

Bob knows what happened from his perspective and Alice knows her version of events.  Where there is a disagreement of what follows from different versions of events it is possible to construct a 3rd person story.  This may be hard to do when you are involved and an actual 3rd person can help but is not crucial in constructing the story.  If you can step outside of your own story and construct a 3rd version together this can resolve misunderstandings.
Something like; "I thought you said we should meet here, even though I said I wanted ice-cream, you thought that meant we should meet at the ice-cream place next door and we each waited 30mins for the other one to turn up to where we were.".  By constructing a 3rd story it's possible that no one was at fault.  It's also possible that it can become clear what went wrong and how to learn from that or what can be done differently.

(cue business management After-Action-Review activities {what did we do well, what could we have done better, what would we do differently}, now SWOT)

4. The Gottman Institute research (and book)

The 4 horsemen of divorce (but just because that's what the research is about doesn't mean we can't apply it elsewhere) (yes Gottman is limited in value because of bad use of statistics we can't be sure the models are accurate, I still find it's a good model at explaining things).

Don't do these things.  When you see these things, recognise them for what they are and don't engage with them.  If necessary acknowledge people are feeling certain angry feelings and let them get them out (not everyone can efficiently drop how they are feeling and get on with talking about it, especially not without practice).

Each one has an antidote, usually in the form of an attitude or strategy that can leave you thinking about the same thing differently and relating to it differently.

I. Criticism
I would rename to "inherent criticism".  Comes in the form of an inherent descriptor like, "you are a lazy person", "you always run late".  "you are the type of person who forgets my birthday"[see 5].  Try to replace inherent criticism with *[6] concrete descriptions of actions.

To counter this - try descriptions like [6a]:  "I can see you are sitting on the couch right now and I would like you to offer help when you can see me cleaning".  "yesterday I saw you try to do a few extra tasks and that caused us to run late", "you forgot my birthday last year".

The important thing about the change here is that an inherent label comes in the form of an unchangeable belief.  It's equivalent to saying, "you are a tall person".  It's fixed in time, space and attitude.  You don't want to give someone a fixed negative trait.  Not in your head and especially not out of your head either to that person or to anyone else.  You set someone up for failure if you do.  As soon as someone is "the lazy one" you give them the ticket to "always be lazy" and if they are half smart they will probably take it.  Besides - you don't change people's actions by using criticism.  You maybe relieve some frustration but then you have created some open frustration and the problem still exists.

II. Defensiveness
Probably easiest to understand by the description of reactive defensiveness.  It usually comes as a reaction to an accusation.  If two people are yelling, chances are neither is listening.  In response to "you are always making us run late", a defensive reaction would be, "I make us run late because you always stress me out".

It does two things:
1. claim to not be responsible
2. make a second accusation (can be irrelevant to the subject at hand).

First of all if you are bringing up several problems at once you are going to confuse matters.  Try to deal with one problem at a time.  It doesn't really matter which so long as you are not yelling about being late while they are yelling about you forgetting the laundry. (and so long as you deal with all the problems)

The second part is that you can't shift blame.  Absorbing some blame does not make you a bad person.  Nor does it make you inherently terrible.  You can have both done a wrong thing and not be a bad person.  After all you had your reasons for doing what you did.

The antidote to defensiveness is to acknowledge [6] what they have said and move forward without reacting.

III. Contempt
This is about an internal state as much as an external state.  Contempt is about the story we tell ourselves about the other person (see NVC) and is a state of negative intent.  I hold you contemptuously.  For example, "a good person would not run late", "if you were smarter you would just...", "I work so hard on this relationship and you just...", Some examples of displays of contempt include when a person uses sarcasm, cynicism, name-calling, eye-rolling, sneering, mockery, and hostile humour [see 7 - emotional intelligence about physiological events].  This overlaps with Inherent criticism and makes more sense with [6 NVC].
Contempt has two antidotes, Teacher mindset and curiosity.  Teacher mindset can change an attitude of, "He should know what he did wrong" to, "I need to explain to him how to do it right".  Curiosity [See NVC, also [3] the 3rd story] can take you to a place of trying to understand what is going on and take you away from the place of the stories we tell ourselves.[10]

IV. Stonewalling
This is a physiological state of going silent.  It is used when you are being lectured (for example) and you go silent, possibly start thinking about everything else while you wait for someone to finish.  It's like holding your breath when you go underwater, waiting for it to pass.  If you are doing this what you need to do is take a break from whatever is going on and do something different, for example go for a walk and calm down.
There was a classic joke, they asked a 110 year old why he lived so long and he said, every time I got into an argument with my wife I used to go for a walk.  I went on a lot of walks in my life.
Because this is a physiological state it's so easy to fix so long as you remember to pay attention to your internal state [see NVC what is most alive in you, and 11. what does that look like in practice]

5. How to win friends and influence people

I always recommend this book to people starting the journey because it's a great place to start.  These days I have better models but when I didn't know anything this was a place to begin.  Most of my models are now more complicated applications of the ideas initially presented.  You still need weak models before replacing them with more complicated ones which are more accurate.
The principles and (in brackets) what has superseded them for me:

1. Don't criticize, condemn or complain. (There are places and methods to do this.  Criticism can be done as [1] from a place of safety or in [4] from a teacher/mentor/growth mindset.  Definitely don't do it from a place of criticism.  Condemnation is more about [10] and is an inherent trait.  Progress doesn't usually happen when we use inherent traits, From Saul Alinsky's rules for radicals - don't complain unless you have the right answer - "I have a problem and you have to figure out how to fix it for me" is not a good way to get your problem solved.)
2. Give honest, sincere appreciation. (so long as you are doing this out of the goodness of your heart good.  If you are using it for manipulation you can just not bother.  NVC supersedes this.  By keeping track of what is most alive in you, you can do better than this)
3. Arouse in the other person an eager want. (Work out what people want, work out how to get both your needs met - superceded by NVC.)
4. Become genuinely interested in other people. (depends what for.  Don't bother if you don't want to.  That would not be genuine.  You need to find the genuine interest inside yourself first.)
5. Smile. (um.  Hard to disagree with but a default smiling state is a good one to cultivate - from [7] physiological states are linked two ways.  Smiling will make you happy just as being happy will make you smile)
6. Remember that a person's name is to that person the most important sound in any language. (I don't know about most important but I would say that anyone can remember names with practice.

7. Be a good listener. Encourage others to talk about themselves. (NVC - pay attention to what is most alive in you when you do. Make sure you know about the spectrum of )
8. Talk in terms of the other person's interest. (Sure why not.  Sales are a lot easier when you are selling what people want. See [15] and NVC to supersede how and why this works)
9. Make the other person feel important - and do so sincerely. (I guess?  I don't do this actively.)
10 The only way to get the best of an argument is to avoid it. ([9] if you are in an argument something already went wrong)
11. Show respect for the other person's opinions. Never say, "You're wrong." (NVC, instead of saying no, say what gets in the way.  "here is evidence that says otherwise" can be better than "durr WRONGGG" but I have seen people use "you are wrong" perfectly fine.)
12. If you are wrong, admit it quickly and emphatically. (hard to disagree with, but holding onto grudges and guity things is not useful.  [4] gottman talks about defensiveness, avoid defensiveness and acknowledge the fact that someone feels you are at fault first.  It will satisfy the psychological need arising in an offended person [14])
13. Begin in a friendly way. (as opposed to what?  Sure I guess.)
14. Get the other person saying, "Yes, yes" immediately. (Yes ladders are important and valuable.  You see bits of this creeping into Gottman [4], NVC [6], The game [13] and other practices but no one as yet explains it as well as I would like.  The game probably has the best commentary on it, short of business books that escape my memory right now)
15. Let the other person do a great deal of the talking. (not really important who talks so long as you are on the same page and in agreement.  If you want someone else to do the emotional labour [15] for you, then you can let them.  If you want to do it for them you can.  Implications of EL are not yet clear to me in full.  Some places it will be good to do EL for people, other places they need to do it for themselves to feel ownership of the problems and solutions)
16. Let the other person feel that the idea is his or hers. (sure I guess.  A good idea is it's own champion.  Ideas that are obviously better will win out.  You can't make a turd beat a diamond but you can employ tricks to polish certain diamonds over others.  This technique is battling over little bits.  can be useful but I would not rely on it alone.)
17. Try honestly to see things from the other person's point of view. (NVC [6] and EL [15] should help do that better.  Imagining that you are that person in a way that is hard to impart in words because it's about having the experience of being that other person (see and not "just thinking about it". needs a longer description and is an effective technique.)
18. Be sympathetic with the other person's ideas and desires. (NVC supercedes.  Everyone has basic feelings and needs that you can understand, like the need for safety)
19. Appeal to the nobler motives. (giving people a reputation to live up to is a valuable technique that I would say only works for qualified people - but does not work so well if you put pressure on people who are less skilled.  Probably relates to the things going through our head at the time - see also book - the inner game of tennis, NVC, judgement model)
20. Dramatize your ideas. (I don't know?  Try it.  It could work.  will not work by virtue of it being a good model of things, might work by luck/breaking people out of their habits)
21. Throw down a challenge. (can work if people are willing to rise to a challenge can work against you and create cognitive dissonance if people are not willing.  Need more information to make it work)
22. Begin with praise and honest appreciation. (Don't give people a shit sandwich - slices of compliments surrounding shit.  That's not respectful of them.  Instead using [1] navigate to a place of safety to talk about things)
23. Call attention to people's mistakes indirectly. (there are correct and incorrect ways to do this.  You can be passive agressive about it.  I don't see a problem with being blunt - in private, in safe conversations [1] - about what is going on)
24. Talk about your own mistakes before criticizing the other person. (don't yammer on, but it can help to connect you and them and the problem.  NVC would be better than just this)
25. Ask questions instead of giving direct orders. (socratic method, can be a drain, need more advanced skills and [15] EL to know if this is appropriate )
26. Let the other person save face. (better described in I agree with this, but [15] EL might describe it better)
27. Praise the slightest and every improvement. Be "lavish in your praise." (NVC disagrees, praise only what is relevant, true and valid.  Be a teacher [4] but deliver praise when praise is due.)
28. Give the other person a fine reputation to live up to. (This is 19/26 again.  I agree with it.  I could use it more)
29. Use encouragement. Make the fault seem easy to correct. (agree, solve the "problem" for someone else, make it easy to move forward)
30. Make the other person happy about doing the thing you suggest. (NVC gives a better model of doing what other people want, "with the joy of a small child feeding a hungry duck")

* Giving people a positive reputation to live up to.  "I trust that you won't forget my birthday again".  Don't be silly with this, "I have confidence that you will give me a million dollars" will not actually yield you a million dollars unless you have reason to believe that will work.

6. NVC - Non-Judgemental communication 

I can't yet do justice to NVC but I am putting together the pieces.  Best to watch the youtube talk in the title link but here are some short points.  Also this helps -
a. Concrete descriptions -
In agreement with Gottman, be concrete and specific -  The objective test of whether the description is concrete is whether the description can be followed by an anonymous person to produce the same experience.  "you are a lazy person" VS "you are sitting on the couch"
b. Acknowledge feelings -
people have huge psychological needs to be heard and understood.  Anyone can fulfill that need
c. Connect that to a need
See the NVC video.
d.  Making a request
See NVC video.
e. Saying no by passing your goals forward
Instead of saying no, Consider what it is that gets in the way of you saying no and say that instead.  Keep in mind vulnerability [16].  This also allows people to plan around your future intentions.  If someone asks you to buy a new car and you say, "no I plan to save money towards buying a house" they can choose to be mindful of that in the future and they can act accordingly (not offering you a different car for sale next week).
f. Connect with what is most alive in you right now
See video for best description.

7. Emotional intelligence

There is a two way path between physiological states and emotional states.

Try these:
a. Hold a pencil/pen in your mouth and go back and read the joke about the old man [4]. (expect to find it funnier than you did the first time)
b. furrow your brow while reading the first paragraph of this page again (expect to either feel confused or the cognitive dissonance version if you know it very well - "I know this too well")
The two way path means that you can feel better about emotional pain by taking a paracetamol, but more specifically, if you take a break from a situation and come back to it the emotions might have improved.  This can include getting a glass of water, going for a walk, getting some fresh air.  And for more complicated decisions - sleeping on it (among other things).

Everyone can train emotional intelligence, they need practice.  This includes holding an understanding of your own states as well as being able to notice emotional states in other people.

I had an ex who had particularly visible physiological states, it was a very valuable experience to me to see the state changes and it really trained my guessing mind to be able to notice changes.  These days I can usually see when things change but I can't always pick the emotion that has come up.  This is where NVC and curiosity become valuable (also circling).

EI is particularly important when it is particularly deficient.  In the book it talks about anger as a state that (to an untrained person) can cause a reaction before someone knows that they were angry.  Make sure to fix that first before moving to higher levels of emotional management.

8. model of arguments

(see also NVC)

If you view disagreements or misunderstandings as a venn diagram of what you know and what the other person knows.  You have full rights to make comment on anything you know but only have limited rights to make comment on what the other person knows.  Instead you can comment on the information they have given you.  "you said 'X', I know Y about what you said 'X'".  To say X is wrong, is not going to yield progress.  Instead to acknowledge that they described 'X' and their description does not make sense to me, or leaves me feeling confused [6].

9. The argument started earlier

From Gavin: "If I ever find myself in a position of saying - well officer, let me explain what happened...", Something already went wrong well and truly before now.
When you start the journey you will start getting to "Aha" moments about where arguments start.  As you get more and more experience you realise the argument started well and truly earlier than you ever first realised.  When you get really good at it, you can stop and say [6] "I am confused"  well and truly before a yelling match.

10. The stories we tell ourselves

NVC based, Judgement model, There is a lot of people who are thinking in stories.  Related -

Their entire existence is the story and narrative they tell about themselves (see also Jordan Peterson - maps of meaning).  The constant narrative about how "the world hates me" is going to give you a particular world experience compared to the constant narrative, "I am a lucky person".  You see this in gamblers who are searching for "the prevailing wind" or "winning streaks".

You also see this in social pressure - when people think and get fixated on, "what will people think of me?", sometimes the social pressure does not even have to be there to cause the thoughts and the actions that would be "social pressure".
Several models of thinking advocate removing the story telling in your head to relieve the psychological pain.  See books, "search inside yourself", NVC, Gateless gatecrashers, some information in the Persistent Non Symbolic Experience Article.

I am not sure what is the best practice, but mindfulness seems to help as well, since these thoughts are all theoretical, grounding yourself in the concrete [6a] and observing those thoughts seems to alleviate the anxieties it can cause.  But this can explain a lot of people's actions (they are telling themselves a particular story in their head).

11. Polling your internal states
[related to 6 NVC]. Any time you are disconnected to what is going on, try asking yourself an internal question of "what is going on?" to connect with what is most alive in you right now.  This might be a feeling of boredom.  It could be anything, but if it's not a good and strong connection with what is presently happening you have a chance to fix it.  (See also the book "The Charisma myth")

12. circling (The circling handbook)

[6 built on NVC] is a practice of living in the current and present experience.  You can focus on another person or focus on yourself.  Perpetually answering the question of "what is most alive in you right now?" and sharing that with other people.

Some examples include:I am feeling nervous sharing this experienceI just closed my eyes and put my head back trying to think of a good example.I am distracted by the sound of birds behind me.I can feel air going past my nostrils as I think about this question.

The creators of cicling find it a very connecting experience to either share what is going on inside you or to guess at what is going on inside someone else and ask if that's an accurate guess.  Or to alternate experiences, each sharing one and one.  or each guessing of each other - one and one.

I find it valuable because everyone can understand present experience, and get a glimpse of your current experience in the process of sharing experience with you.  This method can also work as a form of [15] and [7].

13. The game

(From the book The Game) This concept receives equal part condemnation and praise from various parties.

The basic concept of the game is that life is a game.  Specifically social interactions are a game that you can try out.  You can iterate on and repeat until success.  In the book it follows the journey of a pick up artist as he generally disregards other people's agency and works out how to get what he wants (regularly bed people) through some stages of practicing certain methods of interaction, and iterating until he sees a lot of success.

I see a lot of this concept at kegan stage 3[18].  Everything is about social, and the only thing that matters is social relationships.

Most of the condemnations comes from the failure of this model to treat other people as human, worthy of moral weight, thought or anything other than to be used to your own purposes.  If you don't like dehumanising people the book can still teach you a lot about social interaction, and practicing towards incremental improvement.

If you feel uncomfortable with Pick up, you should examine that belief closely, it's probably to do with feeling uncomfortable with people using manipulation to pursue sex.  That's fine, there is a lot to learn about social and a lot of social systems before you turn into "literally the devil" for knowing about it.  There are also other social goals other than sex that you can pursue.

If you are cautious about turning into a jerk - you are probably not likely to ever even get close to actions that paint you as a jerk because your filters will stop you.  It's the people who have no filter on actions that might want to be careful - herein lies dark arts and being a jerk.  And as much as no one will stop you, no one will really enjoy your presence either if you are a jerk.

The biggest problem I have with game and game methodology is that we all play a one-shot version.  With high stakes of failure.  Which means some of the iteration and having to fail while you learn how to not be terrible - will permanently damage your reputation.  There is no perfect "retry" - a reputation will follow you basically to the ends of the earth and back.  As much as game will teach you some things, the other models in this list have better information for you and are going to go further than game.

14. what an apology must do from Aaron Lazare, M.D.- on apology

1. A valid acknowledgement of the offence that makes clear who the offender is and who is the offended. The offender must clearly and completely acknowledge the offence.
2. An effective explanation, which shows an offence was neither intentional nor personal, and is unlikely to recur.
3. Expressions of remorse, shame, and humility, which show that the offender recognises the suffering of the offended.
4. A reparation of some kind, in the form of a real or symbolic compensation for the offender’s transgression.
An effective apology must also satisfy at least one of seven psychological needs of an offended person.
1. The restoration of dignity in the offended person.
2. The affirmation that both parties have shared values and agree that the harm committed was wrong.
3. Validation that the victim was not responsible for the offense.
4. The assurance that the offended party is safe from a repeat offense.
5. Reparative justice, which occurs when the offended sees the offending party suffer through some type of punishment.
6. Reparation, when the victim receives some form of compensation for his pain.
7. A dialogue that allows the offended parties to express their feelings toward the offenders and even grieve over their losses.

These are not my notes from the book but they are particularly valuable when trying to construct an understanding of apologising and making up for misdeeds.  I don't have them in memory but I know when I need to make a serious apology I can look them up.  They fit quite well with [6], but are more specific to apology and not all interactions.

15. Emotional labour

A relatively new concept.  This is roughly the ability to:
I. Model someone else's emotional state
II. Get it right
III. act on their emotional state

For example:
I. I notice my partners eyes are droopy and they do not appear to be concentrating very well.  Is rubbing eyes and checking their watch a lot.
II. I suspect they are sleepy
III. I make them a coffee, or I offer to make them coffee.  (as a downgraded form I mention they look tired and ask if this is the case)

From Erratio:

Emotional labour is essentially a name for a managerial role in a relationship. This takes on a few different concrete forms.

The first is management of the household, appointments, shopping, and other assorted tasks that are generally shared across couples and/or housemates. Sweeping a floor or cooking dinner is not emotional labour, but being the person who makes sure that those things are accomplished is. It doesn't matter whether you get the floor swept by doing it yourself, asking your partner to do it, firing up a Roomba, or hiring a cleaning service; what matters is that you are taking on responsibility for making sure the task is done. This is why people who say that they would be happy to help with the housework if you would just tell them what needs doing are being a lot less helpful than they think. They're taking the physical labour component of the task but explicitly sticking the other person with the emotional labour component.

The second is taking responsibility for the likes, dislikes, feelings, wants and needs of other people who you are in a relationship with (and to be clear, it doesn't have to be a romantic relationship). Stereotypical scenarios that are covered by this kind of emotional labour include: the hysterical girlfriend who demands that her boyfriend drop everything he's doing to comfort her, the husband who comes home tense and moody after a long day at the office and expects to be asked how his day went and listened to and have validating noises made at him, noticing that the other person in a conversation is uncomfortable and steering the conversation to a more pleasant topic without having to be asked, helping a confused friend talk through their feelings about a potential or former partner, reminding your spouse that it's so-and-so's birthday and that so-and-so would appreciate being contacted, remembering birthdays and anniversaries and holidays and contacting people and saying or doing the right things on each of those dates.

This overlaps with [7].  Commentary on this concept suggest that it's a habit that women get into doing more than men.  Mothers are good at paying attention to their kids and their needs (as the major caregiver from early on), and stemming from this wives also take care of their husbands.  While it would not be fair to suggest that all wives do anything I would be willing to concede that these are habits that people get into and are sometimes socially directed by society.

I am not sure of the overall value of this model but it's clear that it has some implications about how people organise themselves - for better or worse.

16. Vulnerability - Brene brown
In order to form close connections with people a certain level of vulnerability is necessary.  This means that you need to share about yourself in order to give people something to connect to.  In the other direction people need to be a certain level of vulnerable to you in order to connect.  If you make sure to be open and encouraging and not judge you will enable people to open up to you and connect with you.
Sometimes being vulnerable will get you hurt and you need to be aware of that and not shut down future experiences (continue to be open with people).  I see this particularly in people who "take time" to get over relationships.  Being vulnerable is a skill that can be practiced.  Vulnerability replaced a lot of my ideas about [13 The game].  And would have given me a lot of ideas of how to connect with people, combined with [15] and [12]. (I have not read her books but I expect them to be useful)

17. More Than Two (book)

This is commonly known as the polyamory bible.  It doesn't have to be read as a polyamory book, but in the world of polyamory emotional intelligence and the ability to communicate is the bread and butter of every day interactions.  If you are trying to juggle two or three relationships and you don't know how to talk about hard things then you might as well quit now.  If you don't know how to handle difficult feelings or experiences you might as well quit polyamory now.

Reading about these skills and what you might gain from the insight that polyamorous people have learnt is probably valuable to anyone.

18. Kegan stages of development

Other people have summarised this model better than me.  I won't do it justice but if I had to be brief about it - there are a number of levels that we pass through as we grow from very small to more mature.  They include the basic kid level where we only notice inputs and outputs.  Shortly after - when we are sad "the whole world is sad" because we are the whole world.  Eventually we grow out of that and recognise other humans and that they have agency.  At around teenager we end up caring a lot about what other people think about us.  classic teenagers are scared of social pressure and say things like, "I would die if she saw me in this outfit" (while probably being hyperbolic, there is a bit of serious concern present).  Eventually we grow out of that and into system thinking (Libertarian, Socialist, among other tribes).  And later above tribalism into more nuanced varieties of tribes.

It's hard to describe and you are better off reading the theories to get a better idea.  I find the model limited in application but I admit I need to read more about the theories to get my head around it better.

I have a lot more books on the topic to read but I am publishing this list because I feel like I have a good handle on the whole "how people work" and, "how relationships work" thing.  It's rare that anyone does any actions that surprise me (socially) any more.  In fact I am getting so good at it that I trust my intuition [11] more than what people will say sometimes.

When something does not make sense I know what question to ask [6] to get answers.  Often enough it happens that people won't answer the first time, this can represent people not feeling Safe [1] enough to be vulnerable [16].  That's okay.  That represents it's my job to get them to a comfortable place to open up if I want to get to the answers.

I particularly like NVC, Gottman, EL, EI, Vulnerability all of them and find myself using them fortnightly.  Most of these represent a book or more of educational material.  Don't think you know them enough to dismiss them if you have not read the books.  If you feel you know them and already employ the model then it's probably not necessary to look into it further, but if you are ready to dismiss any of these models because they "sound bad" or "don't work" then I would encourage you to do your homework and understand them inside and out before you reject them.

The more models I find the more I find them converging on describing reality.  I am finding less and less I can say, "this is completely new to me" and more and more, "oh that's just like [6] and [7]

Meta: this is something around 6000 words and took a day to write ~12 hours.  I did this in one sitting because everything was already in my head.  I am surprised I could sit still for this long.  (I took breaks for food and a nap but most of today was spent at my desk)

Originally posted on my blog:

Cross posted to Medium:

Idea for LessWrong: Video Tutoring

13 adamzerner 23 June 2017 09:40PM

Update 7/9/17: I propose that Learners individually reach out to Teachers, and set up meetings. It seems like the most practical way of getting started, but I am not sure and am definitely open to other ideas. Other notes:

  • There seems to be agreement that the best way to do this is individualized guidance, rather than lectures and curriculums. Eg. the Teacher "debugging" the Learner. Assuming that approach, it is probably best for the amount of Learners in a session to be small.
  • Consider that it may make sense for you to act as a Teacher, even if you don't have a super strong grasp of the topic. For example, I know a decent amount about computer science, but don't have a super strong grasp of it. Still, I believe it would be valuable for me to teach computer science to others. I can definitely offer value to people with no CS background. And for people who do have a CS background, there could be value in us taking turns teaching/learning, and debugging each other.
  • We may not be perfect at this in the beginning, but let's dive in and see what we can do! I think it'd be a good idea to comment on this post with what did/didn't work for you, so we as a group could learn and improve.
  • I pinned to #productivity on the LessWrongers Slack group.

Update 6/28/17: With 14 people currently interested, it does seem that there's enough to get started. However, I'd like to give it a bit more time and see how much overall interest we get.

Idea: we coordinate to teach each other things via video chat.

  • We (mostly) all like learning. Whether it be for fun, curiosity, a stepping stone towards our goals.
  • My intuition is that there's a lot of us who also enjoy teaching. I do, personally.
  • Enjoyment aside, teaching is a good way of solidifying ones knowledge.
  • Perhaps there would be positive unintended consequences. Eg. socially.
  • Why video? a) I assume that medium is better for education than simply text. b) Social and motivational benefits, maybe. A downside to video is that some may find it intimidating.
  • It may be nice to evolve this into a group project where we iteratively figure out how to do a really good job teaching certain topics.
  • I see the main value in personalization, as opposed to passive lectures/seminars. Those already exist, and are plentiful for most topics. What isn't easily accessible is personalization. With that said, I figure it'd make sense to have about 5 learners per teacher.

So, this seems like something that would be mutually beneficial. To get started, we'd need:

  1. A place to do this. No problem: there's Hangouts, Skype,, etc.
  2. To coordinate topics and times.

Personally, I'm not sure how much I can offer as far as doing the teaching. I worked as a web developer for 1.5 years and have been teaching myself computer science. I could be helpful to those unfamiliar with those fields, but probably not too much help for those already in the field and looking to grow. But I'm interested in learning about lots of things!

Perhaps a good place to start would be to record in some spreadsheet, a) people who want to teach, b) what topics, and c) who is interested in being a Learner. Getting more specific about who wants to learn what may be overkill, as we all seem to have roughly similar interests. Or maybe it isn't.

If you're interested in being a Learner or a Teacher, please add yourself to this spreadsheet.

Instrumental Rationality 1: Starting Advice

13 lifelonglearner 18 June 2017 06:43PM

Starting Advice

[This is the first post in the Instrumental Rationality Sequence. It's a collection of four concepts that I think are central to instrumental rationality—caring about the obvious, looking for practical things, practicing in pieces, and realistic expectations.

Note that these essays are derivative of things I've written here before, so there may not be much new content in this post. (But I wanted to get something out as it'd been about a month since my last update.)

My main goal with this collection was to polish / crystallize past points I've made. If things here are worded poorly, unclear, or don't seem useful, I'd really appreciate feedback to try and improve.]


In Defense of the Obvious:

[As advertised.]

A lot of the things I’m going to go over in this sequence are sometimes going to sound obvious, boring, redundant, or downright tautological. This essay is here to convince you that you should try to listen to the advice anyway, even if it sounds stupidly obvious.

First off, our brains don’t always see all the connections at once. Thus, even if some given advice is apparentlyobvious, you still might be learning things.

For example, say someone tells you, “If you want to exercise more, then you should probably exercise more. Once you do that, you’ll become the type of person who exercises more, and then you’ll likely exercise more.”

The above advice might sound pretty silly, but it may still be useful. Often, our mental categories for “exercise” and “personal identity” are in different places. Sure, it’s tautologically true that someone who exercises becomes a person who exercises more. But if you’re not explicitly thinking about how your actions change who you are, then there’s likely still something new to think about.

Humans are often weirdly inconsistent with our mental buckets—things that logically seem like they “should” be lumped together often aren't. By paying attention to even tautological advice like this, you’re able to form new connections in your brain and link new mental categories together, perhaps discovering new insights that you “already knew”.

Secondly, obvious advice tends to be low-hanging fruit. If your brain is pattern-matching something as “boring advice” or “obvious”, you’ve likely heard it before many times before.

For example, you can probably guess the top 5 things on any “How to be Productive” list—make a schedule, remove distractions, take periodic breaks, etc. etc. You can almost feel your brain roll its metaphorical eyes at such dreary, well-worn advice.

But if you’ve heard these things repeated many times before, this is also good reason to suspect that, at least for a lot of people, it works. Meaning that if you aren’t taking such advice already, you can probably get a boost by doing so.

If you just did those top 5 things, you’d probably already be quite the productive person.

The trick, then, is to actually do them. That means doing the obvious thing.


Lastly, it can be easy to discount obvious advice when you’ve seen too much of it. When you’re bombarded with boring-seeming advice from all angles, it’s easy to become desensitized.

What I mean is that it’s possible to dismiss obvious advice outright because it sounds way too simple. “This can’t possibly work,” your brain might say, “The secret to getting things done must be more complex!”

There’s something akin to the hedonic treadmill happening here where, after having been exposed to all the “normal” advice, you start to seek out deeper and deeper ideas in search of some sort of mental high. What happens is that you become a kind of self-help junkie.

You can end up craving the bleeding edge of crazy ideas because literally nothing else seems worthwhile. You might end up dismissing normal helpful ideas simply because they’re not paradigm-crushing, mind-blowing, or mentally stimulating enough.

At which point, you’ve adopted quite the contrarian stance—you reject the typical idea of advice on grounds of its obviousness alone.

If this describes, might I tempt you with the meta-contrarian point of view?

Here’s the sell: One of the secrets to winning at life is looking at obvious advice, acknowledging that it’s obvious, and then doing it anyway.

(That’s right, you can join the elite group of people who scoff at those who scoff at the obvious!)

You can both say, “Hey, this is pretty simple stuff I’ve heard a thousand times before,” as well as say, “Hey, this is pretty useful stuff I should shut up and do anyway even if it sounds simple because I’m smart and I recognize the value here.”

At some point, being more sophisticated than the sophisticates means being able the grasp the idea that not all things have to be hyper complex. Oftentimes, the trick to getting something done is simply to get started and start doing it.

Because some things in life really are obvious.

Hunting for Practicality:

[This is about looking for ways to have any advice you read be actually useful, by having it apply to the real world. ]

Imagine someone trying to explain exactly what the mitochondria does in the cell, and contrast that to someone trying to score a point in a game of basketball.

There’s something clearly different about what each person is trying to do, even if we lumped both under the label of “learning” (one is learning about cells and the other is learning about basketball).

In learning, it turns out this  divide is often separated into declarative and procedural knowledge.

Declarative knowledge is like the student trying to puzzle out the ATP question; it’s about what you know.

In contrast, procedural knowledge, like the fledgling basketball player, is about what you do.

I bring up this divide because many of the techniques in instrumental rationality will feel like declarative knowledge, but they’ll really be procedural in nature.

For example, say you’re reading something on motivation, and you learn that “Motivation = Energy to do the thing + a Reminder to do the thing + Time to do the thing = E+R+T”.

What’ll likely happen is that your brain will form a new set of mental nodes that connects “motivation” to “E+R+T”. This would be great if I ended up quizzing you “What does motivation equal?” whereupon you’d correctly answer “E+R+T”.

But that’s not the point here! The point is to have the equation actually cash out into the real world and positively affect your actions. If information isn’t changing you view or act, then you’re probably not extracting all the value you can.

What that means is figuring out the answer to this question: "How do I see myself acting differently in the future as a result of this question?"

With that in mind, say you generate some examples and make a list.

Your list of real-world actions might end up looking like:

1) Remembering to stay hydrated more often (Energy)

2) Using more Post-It notes as memos (Reminder)

3) Start using Google Calendar to block out chunks of time (Time).

The point is to be always on the lookout for ways to see how you can use what you’re learning to inform your actions. Learning about all these things is only useful if you can find ways to apply them. You want to do more than have empty boxes that link concepts together. It’s important to have those boxes linked up to ways you can do better in the real world.

You want to actually put in some effort trying to answer question of practicality.




Actually Practicing:

[This is about knowing the nuances of little steps behind any sort of self-improvement skill you learn, and how those little steps are important when learning the whole.]

So on one level, using knowledge from instrumental rationality is about how you take declarative-seeming information and find ways to actually get real-world actions out of it. That’s important.

But it’s also important to note that the very skill of “Generating Examples”—the thing you did in the above essay to even figure out which actions can fit in the above equation to fill in the blanks of E, R, and T—is itself a mental habit that requires procedural knowledge.

What I mean is that there’s a subtler thing that’s happening inside your head when you try to come up with examples—your brain is doing something—and this “something” is important.  

It’s important, I claim, because if we peer a little more deeply at what it means for your brain to generate examples, we’ll come away with a list of steps that will feel a lot like something a brain can do, a prime example of procedural knowledge.

For example, we can imagine a magician trying to learn a card trick. They go through the steps. First they need to spread the cards. Then comes the secret move. Finally comes the final reveal of the selected card in the magician’s pocket.

What the audience member sees is the full finished product. And indeed, the magician who’s practiced enough will also see the same thing. But it’s not until the magician goes through all the steps and understands how all the steps flow together to form the whole card trick that they’re ready to perform.

The idea here is to describe any mental skill with enough granularity and detail, at the 5 second level, such that you’d both be able to go through the same steps a second time and teach someone else. So being able to take skills and chunk them into smaller pieces is also forms another core part of learning.


Realistic Expectations:

[An essay about having realistic expectations and looking past potentially harmful framing effects.]

There’s this tendency to get frustrated with learning mental techniques after just a few days. I think this is because people miss the declarative vs procedural distinction. (But you hopefully won’t fall prey to it because we’ve covered the distinction now!)

Once we liken the analogy to be more like that playing a sport, it becomes much easier to see that any expectation of immediately learning a mental habit is rather silly—no one expects to master tennis in just a week.

So, when it comes to trying to configure your expectations, I suggest that you try to renormalize your expectations by treating learning mental habits more like learning a sport.

Keep that as an analogy, and you’ll likely get fairly well-calibrated expectations for learning all this stuff.

Still, what, then, might be a realistic time frame for learning?

We’ll go over habits in far more detail in a later section, but a rough number for now is approximately two months. You can expect that, on average, it’ll take you about 66 days to ingrain a new habit.

Similarly, instrumental rationality (probably) won’t make you a god. In my experience, studying these areas has been super useful, which is why I’m writing at all. But I would guess that, optimistically, I only about doubled my work output.

Of course your own mileage may vary depending where you are right now, but this serves as the general disclaimer to keep your expectations within the bound of reality.

Here, the main point is that, even though mental habits don’t seem like they should be more similar to playing a sport, they really are. There’s something here about how first impressions can be rather deceiving.

For example, a typical trap I might fall into is missing the distinction between “theoretically possible” and “realistic”. I end up looking at the supposed 24 hours available to me everyday and then beating myself up for not being able to harness all 24 hours to do productive work.

But such a framing of the situation is inaccurate; things like sleep and eating are often very essential to maximizing productivity for the rest of the hours! So when diving in and practicing, try to look a little deeper when setting your expectations.


The Rationalistsphere and the Less Wrong wiki

13 Deku-shrub 12 June 2017 11:29PM

Hi everyone!

For people not acquainted with me, I'm Deku-shrub, often known online for my cybercrime research, as well as fairly heavy involvement in the global transhumanist movement with projects like the UK Transhumanist Party and the H+Pedia wiki.

For almost 2 years year now on and off I have been trying to grok what Less Wrong is about, but I've shirked reading all the sequences end to end, instead focused on the most popular ideas transmitted by Internet cultural osmosis. I'm an amateur sociologist and understanding Less Wrong falls within my wider project of understanding the different trends within the contemporary and historical transhumanist movement.

I'm very keen to pin down today's shape of the rationalistsphere and its critics, and the best place I have found do this is on the wiki. Utilising Cunningham's Law at times, I've been building some key navigational and primer articles on the wiki. However with the very lowest hanging fruit now addressed I ask - what next for the wiki?

Distillation of Less Wrong

There was a historical attempt to summerise all major Less Wrong posts, an interesting but incomplete project. It was also approach without a usefully normalised approach. Ideally, every article would have its own page which could be heavily tagged up with metadata such a themes, importance, length, quality, author and such. Is this the goal of the wiki?

Outreach and communications

Another major project is to fully index the Diaspora across Twitter, Blogs, Tumblr, Reddit, Facebook etc and improve the flow of information between the relevant sub communities.

You'll probably want to join one of the chat platforms if you're interested in getting involved. Hell, there are even a few memes and probably more to collect.

Rationalist research

I'll admit I'm ignorant of the goal of Arbital, but I do love me a wiki for research. Cross referencing and citing ideas, merging, splitting, identifying and fully capturing truly interesting and useful ideas from fanciful and fleeting ones is how I've become an expert in a number fields, just by being the first to assemble All The Things.

Certain ideas like the Paper clip maximizer have some popularity beyond just Less Wrong, but Murder Gandhi doesn't - yet. Polishing these ideas with existing and external references (and maybe blogging about them?) is a great way for the community discussion of yore to make its way into the publications of lazy journalists for dissemination. Hell, RationalWiki has been doing it for years now, they're not the only game in town.


If you have any ideas in these areas, or others just a technical, let me know either here, on the Less Wrong Slack group, or on my talk page and maybe we can make Wikis Great Again? ;)

We are the Athenians, not the Spartans

13 wubbles 11 June 2017 05:53AM

The Peloponnesian War was a war between two empires: the seadwelling Athenians, and the landlubber Spartans. Spartans were devoted to duty and country, living in barracks and drinking the black broth. From birth they trained to be the caste dictators of a slaveowning society, which would annually slay slaves to forestall a rebellion. The most famous Spartan is Leonidas, who died in a heroic last stand delaying the invasion of the Persians. To be a Spartan was to live a life devoted to toughness and duty.

Famous Athenians are Herodotus, inventor of history, Thucydides, Socrates, Plato, Hippocrates of the oath medical students still take, all the Greek playwrights, etc.  Attic Greek is the Greek we learn in our Classics courses. Athens was a city where the students of the entire known Greek world would come to learn from the masters, a maritime empire with hundreds of resident aliens, where slavery was comparable to that of the Romans. Luxury apartments, planned subdivisions, sexual hedonism, and free trade made up the life of the Athenian elite.

These two cities had deeply incompatible values. Spartans lived in fear that the Helots would rebel and kill them. Deeply suspicious of strangers, they imposed oligarchies upon the cities they conquered. They were described by themselves and others cautious and slow to act. Athenians by contrast prized speed and risk in their enterprises. Foreigners could live freely in Athens and even established their own temples. Master and slave comedies of Athens inspired PG Woodhouse.

All intellectual communities are Athenian in outlook. We remember Sparta for its killing and Athens for its art. If we want the rationalist community to tackle the hard problems, if we support a world that is supportive of human values and beauty, if we yearn to end the plagues of humanity, our values should be Athenian: individualistic, open, trusting, enamoured of beauty. When we build social technology, it should not aim to cultivate values that stand against these.

High trust, open, societies are the societies where human lives are most improved. Beyond merely being refugees for the persecuted they become havens for intellectual discussion and the improvement of human knowledge and practice. It is not a coincidence that one city produced Spinoza, Rubens, Rembrandt, van Dyke, Huygens, van Leeuwenhoek, and Grotius in a few short decades, while dominating the seas and being open to refugees.

Sadly we seem to have lost sight of this in the rationality community. Increasingly we are losing touch as a community with the outside intellectual world, without the impetus to study what has been done before and what the research lines are in statistics, ML, AI, epistemology, biology, etc. While we express that these things are important, the conversation doesn't seem to center around the actual content of these developments. In some cases (statistics) we're actively hostile to understanding some of the developments and limitations of our approach as a matter of tribal marker.

Some projects seem to me to be likely to worsen this, either because they express Spartan values or because they further physical isolation in ways that will act to create more small-group identification.

What can we do about this? Holiday modifications might help with reminding us of our values, but I don't know how we can change the community's outlook more directly. We should strive to stop merely acting on the meta-level and try to act on the object level more as a community. And lastly, we should notice that our values are real and not universal, and that they need defending.

A new, better way to read the Sequences

13 SaidAchmiz 04 June 2017 05:10AM

A new way to read the Sequences:

It's also more mobile-friendly than a PDF/mobi/epub.

(The content is from the book — Rationality: From AI to Zombies. Books I through IV are up already; Books V and VI aren't up yet, but soon will be.)

Edit: Book V is now up.

Edit 2: Book VI is now up.

Edit 3: A zipped archive of the site (for offline viewing) is now available for download.

Announcing AASAA - Accelerating AI Safety Adoption in Academia (and elsewhere)

12 toonalfrink 15 June 2017 06:55PM

AI safety is a small field. It has only about 50 researchers, and it’s mostly talent-constrained. I believe this number should be drastically higher.

A: the missing step from zero to hero

I have spoken to many intelligent, self-motivated people that bear a sense of urgency about AI. They are willing to switch careers to doing research, but they are unable to get there. This is understandable: the path up to research-level understanding is lonely, arduous, long, and uncertain. It is like a pilgrimage.

One has to study concepts from the papers in which they first appeared. This is not easy. Such papers are undistilled. Unless one is lucky, there is no one to provide guidance and answer questions. Then should one come out on top, there is no guarantee that the quality of their work will be sufficient for a paycheck or a useful contribution.

Unless one is particularly risk-tolerant or has a perfect safety net, they will not be able to fully take the plunge.
I believe plenty of measures can be made to make getting into AI safety more like an "It's a small world"-ride:

  • Let there be a tested path with signposts along the way to make progress clear and measurable.

  • Let there be social reinforcement so that we are not hindered but helped by our instinct for conformity.

  • Let there be high-quality explanations of the material to speed up and ease the learning process, so that it is cheap.

B: the giant unrelenting research machine that we don’t use

The majority of researchers nowadays build their careers through academia. The typical story is for an academic to become acquainted with various topics during their study, pick one that is particularly interesting, and work on it for the rest of their career.

I have learned through personal experience that AI safety can be very interesting, and the reason it isn’t so popular yet is all about lack of exposure. If students were to be acquainted with the field early on, I believe a sizable amount of them would end up working in it (though this is an assumption that should be checked).

AI safety is in an innovator phase. Innovators are highly risk-tolerant and have a large amount of agency, which allows them to survive an environment with little guidance, polish or supporting infrastructure. Let us not fall for the typical mind fallacy, expecting less risk-tolerant people to move into AI safety all by themselves. Academia can provide that supporting infrastructure that they need.

AASAA adresses both of these issues. It has 2 phases:

A: Distill the field of AI safety into a high-quality MOOC: “Introduction to AI safety”

B: Use the MOOC as a proof of concept to convince universities to teach the field




We are bottlenecked for volunteers and ideas. If you'd like to help out, even if just by sharing perspective, fill in this form and I will invite you to the slack and get you involved.

Existential risk from AI without an intelligence explosion

12 AlexMennen 25 May 2017 04:44PM

[xpost from my blog]

In discussions of existential risk from AI, it is often assumed that the existential catastrophe would follow an intelligence explosion, in which an AI creates a more capable AI, which in turn creates a yet more capable AI, and so on, a feedback loop that eventually produces an AI whose cognitive power vastly surpasses that of humans, which would be able to obtain a decisive strategic advantage over humanity, allowing it to pursue its own goals without effective human interference. Victoria Krakovna points out that many arguments that AI could present an existential risk do not rely on an intelligence explosion. I want to look in sightly more detail at how that could happen. Kaj Sotala also discusses this.

An AI starts an intelligence explosion when its ability to create better AIs surpasses that of human AI researchers by a sufficient margin (provided the AI is motivated to do so). An AI attains a decisive strategic advantage when its ability to optimize the universe surpasses that of humanity by a sufficient margin. Which of these happens first depends on what skills AIs have the advantage at relative to humans. If AIs are better at programming AIs than they are at taking over the world, then an intelligence explosion will happen first, and it will then be able to get a decisive strategic advantage soon after. But if AIs are better at taking over the world than they are at programming AIs, then an AI would get a decisive strategic advantage without an intelligence explosion occurring first.

Since an intelligence explosion happening first is usually considered the default assumption, I'll just sketch a plausibility argument for the reverse. There's a lot of variation in how easy cognitive tasks are for AIs compared to humans. Since programming AIs is not yet a task that AIs can do well, it doesn't seem like it should be a priori surprising if programming AIs turned out to be an extremely difficult task for AIs to accomplish, relative to humans. Taking over the world is also plausibly especially difficult for AIs, but I don't see strong reasons for confidence that it would be harder for AIs than starting an intelligence explosion would be. It's possible that an AI with significantly but not vastly superhuman abilities in some domains could identify some vulnerability that it could exploit to gain power, which humans would never think of. Or an AI could be enough better than humans at forms of engineering other than AI programming (perhaps molecular manufacturing) that it could build physical machines that could out-compete humans, though this would require it to obtain the resources necessary to produce them.

Furthermore, an AI that is capable of producing a more capable AI may refrain from doing so if it is unable to solve the AI alignment problem for itself; that is, if it can create a more intelligent AI, but not one that shares its preferences. This seems unlikely if the AI has an explicit description of its preferences. But if the AI, like humans and most contemporary AI, lacks an explicit description of its preferences, then the difficulty of the AI alignment problem could be an obstacle to an intelligence explosion occurring.

It also seems worth thinking about the policy implications of the differences between existential catastrophes from AI that follow an intelligence explosion versus those that don't. For instance, AIs that attempt to attain a decisive strategic advantage without undergoing an intelligence explosion will exceed human cognitive capabilities by a smaller margin, and thus would likely attain strategic advantages that are less decisive, and would be more likely to fail. Thus containment strategies are probably more useful for addressing risks that don't involve an intelligence explosion, while attempts to contain a post-intelligence explosion AI are probably pretty much hopeless (although it may be worthwhile to find ways to interrupt an intelligence explosion while it is beginning). Risks not involving an intelligence explosion may be more predictable in advance, since they don't involve a rapid increase in the AI's abilities, and would thus be easier to deal with at the last minute, so it might make sense far in advance to focus disproportionately on risks that do involve an intelligence explosion.

It seems likely that AI alignment would be easier for AIs that do not undergo an intelligence explosion, since it is more likely to be possible to monitor and do something about it if it goes wrong, and lower optimization power means lower ability to exploit the difference between the goals the AI was given and the goals that were intended, if we are only able to specify our goals approximately. The first of those reasons applies to any AI that attempts to attain a decisive strategic advantage without first undergoing an intelligence explosion, whereas the second only applies to AIs that do not undergo an intelligence explosion ever. Because of these, it might make sense to attempt to decrease the chance that the first AI to attain a decisive strategic advantage undergoes an intelligence explosion beforehand, as well as the chance that it undergoes an intelligence explosion ever, though preventing the latter may be much more difficult. However, some strategies to achieve this may have undesirable side-effects; for instance, as mentioned earlier, AIs whose preferences are not explicitly described seem more likely to attain a decisive strategic advantage without first undergoing an intelligence explosion, but such AIs are probably more difficult to align with human values.

If AIs get a decisive strategic advantage over humans without an intelligence explosion, then since this would likely involve the decisive strategic advantage being obtained much more slowly, it would be much more likely for multiple, and possibly many, AIs to gain decisive strategic advantages over humans, though not necessarily over each other, resulting in a multipolar outcome. Thus considerations about multipolar versus singleton scenarios also apply to decisive strategic advantage-first versus intelligence explosion-first scenarios.

Book Review: Mathematics for Computer Science (Suggestion for MIRI Research Guide)

11 richard_reitz 22 July 2017 07:26PM

tl;dr: I read Mathematics for Computer Science (MCS) and found it excellent. I sampled Discrete Mathematics and Its Applications (Rosen)—currently recommended in MIRI's research guide—as well as Concrete Mathematics and Discrete Mathematics with Applications (Epp), which appear to be MCS's competition. Based on these partial readings, I found MCS to be the best overall text. I therefore recommend MIRI change the recommendation in its research guide.


MCS is used at MIT for their introductory discrete math course, 6.042, which appears to be taken primarily by second-semester freshman and sophomores. You can find OpenCourseWare archives from 2010 and 2015, although the book is self-contained; I never had occasion to use them throughout my reading. 

If you liked Computability and Logic (review), currently in the MIRI research guide, you'll like MCS:

MCS is a wonderful book. It's well written. It's rigorous, but does a nice job of motivating the material. It efficiently proves a number of counterintuitive results and then helps you see them as intuitively obvious. Freed from the constraint of printing cost, it contains many diagrams which are generally useful. You can find the pdf here or, if that link breaks, by googling "Mathematics for Computer Science". (See section 21.2 for why this works.)

MCS is regularly updated during the semester. Based on the dates of revision given to the cover, I suspect that the authors attempt to update it within a week of the last update during the semester. The current version is 87 pages longer than the 2015 version, suggesting ~40 pages of material is added a year. My favorite thing about the constant updates was that I never needed to double check statements about our current state of knowledge to see if anything had changed since publication.

MCS is licensed under a Creative Commons attribution share-alike license: it is free in the sense of both beer and freedom. I'm a big fan of such copyleft licenses, so I give MIT major props. I've tried to remain unbiased in my review, but halo effect suggests my views on the text might be affected by the text's license: salt accordingly.


The only prerequisite is single-variable calculus. In particular, I noted integration, differentiation, and convergence/infinite sums coming up. That said, I don't remember seeing them coming up in sections that provided a lot of dependencies: with just a first course in algebra, I feel a smart 14-year-old could get through 80–90% of the book, albeit with some help, mostly in places where "do a bunch of algebra" steps are omitted. An extra 4–5 years of practice doing algebraic manipulations makes a difference.

MCS is also an introduction to proofwriting. In my experience, writing mathematical proofs is a skill complex enough to require human feedback to get all the nuances of why something works and why something else doesn't work and why one approach is better than another. If you've never written proofs before and would like a human to give you feedback, please pm me.

Comparison to Other Discrete Math Texts


I randomly sampled section 4.3 of Rosen, on primes and greatest common divisors and was very unimpressed. Rosen states the fundamental theorem of arithmetic without a proof. The next theorem had a proof which was twice as long and half as elegant as it could have been. The writing was correct but unmotivating and wordy. For instance, Rosen writes "If n is a composite integer", which is redundant, since all composite numbers are integers, so he could have just said "If n is composite".

In the original Course Recommendations for Friendliness Researchers, Louie responded to Rosen's negative reviews:

people taking my recommendations would be geniuses by-and-large and that the harder book would be better in the long-run for the brightest people who studied from it.

Based on the sample I read, Rosen is significantly dumbed-down relative to MCS. Rosen does not prove the fundamental theorem of arithmetic whereas MCS proves it in section 9.4. For the next theorem, Rosen gives an inelegant proof when a much sleeker—but reasonably evident!—proof exists, making it feel like Rosen expected the reader to not be able to follow the sleeker proof. Rosen's use of "composite integer" instead of "composite" seems like he assumes the reader doesn't understand that the only objects one describes as composite are integers; MCS does not contain the string "composite integer".

In the section I read, Rosen has worked examples for finding gcd(24, 36) and gcd(17, 22), something I remember doing when I was 12. It's almost like Rosen was spoon-feeding how to guess the teacher's password for the student to regurgitate on an exam instead of building insight.

Concrete Mathematics

There are probably individuals who would prefer Concrete Mathematics to MCS. These people are probably into witchcraft.

I explain by way of example. In section 21.1.1, MCS presents a very sleek, but extremely nonobvious, proof of gambler's ruin using a clever argument courtesy of Pascal. In section 21.1.2, MCS gives a proof that doesn't require the reader to be "as ingenuious Pascal [sic]". As an individual who is decidedly not as ingenious as Pascal was, I appreciate this.

More generally, say we want to prove a theorem that looks something like "If A, then B has property C." You start at A and, appealing to the definition of C, show that B has it. There's probably some cleverness involved in doing so, but you start at the obvious place (A), end in the obvious place (B satisfies the definition of C), and don't rely on any crazy, seemingly-unrelated insights. Let's call this sort of proof mundane.

(Note that mundane is far from mechanical. Most of the proofs in Baby Rudin are mundane, but require significant cleverness and work to generate independently.)

There is a virtue in mundane proofs: a smart reader can usually generate them after they read the theorem but before they read its proof. Doing is beneficial, since proof-generating makes the theorem more memorable. It also gives the reader practice building intuition by playing around with the mathematical objects and helps them improve their proofwriting by comparing their output to a maximally refined proof.

On the end of the spectrum opposing mundane is witchcraft. Proofs that use witchcraft typically have a step where you demonstrate you're as ingenious as Pascal by having a seemingly-unrelated insight that makes everything easier. Notice that, even if you are as ingenious as Pascal, you won't necessarily be able to generate these insights quickly enough to get through the text at any reasonable pace.

For the reasons listed above, I prefer mundane proofs. This isn't to say MCS is devoid of witchcraft: sometimes it's the best or only way of getting a proof. The difference is that MCS uses mundane proofs whenever possible whereas Concrete Mathematics invokes witchcraft left and right. This is why I don't recommend it.

Individuals who are readily as ingenious as Pascal, don't want the skill-building benefits of mundane proofs, or prefer the whimsy of witchcraft may prefer Concrete Mathematics.


I randomly sampled section 12.2 of Epp and found it somewhat dry but wholly unobjectionable. Unlike Rosen, I felt like Epp was writing for an intelligent human being (though I was reading much further along in the book, so maybe Rosen assumed the reader was still struggling with the idea of proof). Unlike Concrete Mathematics, I detected no witchcraft. However, I felt that Epp had inferior motivation and was written less engagingly. Epp is also not licensed under Creative Commons.


Epp, Rosen, and MCS are all ~1000 pages long, whereas Concrete Mathematics is ~675. To determine what these books covered that might not be in MCS, I looked through their table of contents' for things I didn't recognize. The former three have the same core coverage, although Epp and Rosen go into material you would find in Computability and Logic or Sipser (also part of the research guide), whereas MCS spends more time developing discrete probability. Based on the samples I read, Epp and MCS have about the same density, whereas Rosen spends little time building insight and a lot of time showing how to do really basic, obvious stuff. I would expect Epp and MCS to have roughly the same amount of content covering mostly (but not entirely) the same stuff and Rosen to offer a mere shadow of the insight of the other two.

Concrete Mathematics seems to contain a subset of MCS's topics, but from the sections I read, I expect the presentation to be wildly different.


My only substantial complaint about MCS is that, to my knowledge, the source LaTeX is not available. Contrast this to SICP, which has the HTML available. This resulted in a proliferation of PDFs tailored for different use cases. It'd be nice, for instance, to have a print-friendly version of MCS (perhaps with fewer pages), plus a version that fit nicely onto the small screen of an ereader or mobile device, plus a version with the same aspect ratio as my monitor. This all would be extremely easy to generate given the source. It would also facilitate crowdsourcing proofreading: there are more than a few typos, although they don't preempt comprehension. At the very least, I wish there were somewhere to submit errata.

Some parts of MCS were notation-heavy. To quote what a professor once wrote on a problem set of mine:

I'm not sure all the notation actually serves the goal of clarifying the argument for the reader. Of course, such notation is sometimes needed. But when it is not needed, it can function as a tool with which to bludgeon the reader…

I found myself referring to Wikipedia's glossary of graph theory terms more than a few times when I was making definitions to put into Anki. Not sure if this is measuring a weak section or a really good glossary or something else.

A Note on Printing

A lot of people like printed copies of their books. One benefit of MCS I've put forward is that it's free (as in beer), so I investigated how much printing would cost.

I checked the local print shops and Kinko's online was unable to find printing under $60, a typical price around $70, with the option to burn $85 if I wanted nicer paper. This was more than I had expected and between ⅓ and ½ (ish) the price of Rosen or Epp.

Personally, I think printing is counterproductive, since the PDF has clickable links.

Final Thoughts

Despite sharing first names, I am not Richard Stallman. I prefer the license on MCS to the license on its competitors, but I wouldn't recommend it unless I thought the text itself was superior. I would recommend baby Rudin (nonfree) over French's Introduction to Real Analysis; Hoffman and Kunze's Linear Algebra (nonfree) over Jim Hefferson's Linear Algebra; and Epp over 2010!MCS. The freer the better, but that consideration is trumped by the quality of the text. When you're spending >100 hours working out of a book that provides foundational knowledge for the rest of your life, ~$150 and a loss of freedom is a price many would pay for better quality.

Eliezer writes:

Tell a real educator about how Earth classes are taught in three-month-sized units, and they would’ve sputtered and asked how you can iterate fast enough to learn how to teach that.

Rosen is in its seventh edition. Epp is in its fourth edition and Concrete Mathematics its second. The earliest copy of MCS I've happened across comes from 2004. Near as I can tell, it is improved every time the authors go through the material with their students, which would put it in its 25th edition.

And you know what? It's just going to keep getting better faster than anything else.


Thank you to Gram Stone for reviewing drafts of this review.

LessWrong Is Not about Forum Software, LessWrong Is about Posts (Or: How to Immanentize the LW 2.0 Eschaton in 2.5 Easy Steps!)

11 enye-word 15 July 2017 09:35PM

[epistemic status: I was going to do a lot of research for this post, but I decided not to as there are no sources on the internet so I'd have to interview people directly and I'd rather have this post be imperfect than never exist.]

Many words have been written about how LessWrong is now shit. Opinions vary about how shit exactly it is. I refer you to and for more comments about LessWrong being shit and the LessWrong diaspora being suboptimal.

However, how to make LessWrong stop being shit seems remarkably simple to me. Here are the steps to resurrect it:

1. Get Eliezer: The lifeblood of LessWrong is Eliezer Yudkowsky's writing. If you don't have that, what's the point of being on this website? Currently Eliezer is posting his writings on Facebook, ( which I consider foolish, for the same reasons I would consider it foolish to house the Mona Lisa in a run-down motel.

2. Get Scott: Once you have Eliezer back, and you sound the alarm that LW is coming back, I'm fairly certain that Scott "Yvain" Alexander will begin posting on LessWrong again. As far as I can tell he's never wanted to have to moderate a comment section, and the growing pains are stressing his website at the seams. He's even mused publicly about arbitrarily splitting the Slate Star Codex comment section in two ( which is a crazy idea on its own but completely reasonable in the context of (cross)posting to LW. Once you have Yudkowsky and Yvain, you have about 80% of what made LessWrong not shit.

3. Get Gwern: I don't read many of Gwern's posts; I just like having him around. Luckily for us, he never left!

After this is done, everyone else should wander back in, more or less.

Possible objections, with replies:

Objection: Most SSC articles and Yudkowsky essays are not on the subject of rationality and thus for your plan to work LessWrong's focus would have to subtly shift.

Reply: Shift away, then! It's LessWrong 2! We no longer have to be a community dedicated to reading Rationality: From AI to Zombies as it's written in real time; we can now be a community that takes Rationality: From AI to Zombies as a starting point and discusses whatever we find interesting! Thus the demarcation between 1.0 and 2.0!

Objection: People on LessWrong are mean and I do not like them.

Reply: The influx of new readers from the Yudkowsky-Yvain in-migration should make the tone on this website more upbeat and positive. Failing that, I don't know, ban the problem children, I guess. I don't know if it's poor form to declare this but I'd rather have a LessWrong Principate than a LessWrong Ruins. See also:

Objection: I'd prefer, for various reasons, to just let LessWrong die.

Reply: Then kill it with your own hands! Don't let it lie here on the ground, bleeding out! Make a post called "The discussion thread at the end of the universe" that reads "LessWrong is over, piss off to r/SlateStarCodex", disallow new submissions, and be done with it! Let it end with dignity and bring a close to its history for good.

Bi-Weekly Rational Feed

11 deluks917 09 July 2017 07:11PM

===Highly Recommended Articles:

Just Saying What You Mean Is Impossible by Zvi Moshowitz - "Humans are automatically doing lightning fast implicit probabilistic analysis on social information in the background of every moment of their lives." This implies there is no way to divorce the content of your communication from its myriad probabilistic social implications. Different phrasings will just send different implications.

In Defense Of Individualist Culture by Sarah Constantin (Otium) - A description of individualist culture. Criticisms of individualist culture: Lacking sympathy, few good defaults. Defenses: Its very hard to change people (psychology research review). A defense of naive personal identity. Traditional culture is fragile. Building a community project is hard in the modern world, prepare for the failure modes. Modernity has big upsides, some people will make better choices than the traditional rules allow.

My Current Thoughts On Miris Highly Reliable by Daniel Dewey (EA forum) - Report by the Open Phil AI safety lead. A basic description of and case for the MIRI program. Conclusion: 10% credence in MIRI's work being highly useful. Reasons: Hard to apply to early agents, few researchers are excited, other approaches seem more promising.

Conversation With Dario Amodei by Jeff Kaufman - "The research that's most valuable from an AI safety perspective also has substantial value from the perspective of solving problems today". Prioritize work on goals. Transparency and adversarial examples are also important.

Cfar Week 1 by mindlevelup - What is working at CFAF actually like. Less rationality research than anticipated. Communication costs scale quadratically. Organization efficiency and group rationality.

The Ladder Of Interventions by mindlevelup - "This is a hierarchy of techniques to use for in-the-moment situations where you need to “convince” yourself to do something." The author uses these methods in practice.

On Dragon Army by Zvi Moshowitz - Long response to many quotes from "Dragon Army Barracks". Duncan't attitude to criticism. Tyler Durden shouldn't appeal to Duncan. Authoritarian group houses haven't been tried. Rationalists undervalue exploration. Loneliness and doing big things. The pendulum model of social progress. Sticking to commitments even when its painful. Saving face when you screw up. True Reliability: The bay is way too unreliable but Duncan goes too far. Trust and power Dynamics. Pragmatic criticism of the charter.

Without Belief In A God But Never Without Belief In A Devil by Lou (sam[]zdat) - The nature of mass movements. The beats and the John Birchers. The taxonomy of the frustrated. Horseshoe theory. The frustrated cannot derive satisfaction from action, something else has to fill the void Poverty, work and meaning. Mass movements need to sow resentment. Hatred is the strongest unifier. Modernity inevitably causes justified resentment. Tocqueville, Polyanai, Hoffer and Scott's theories. Helpful and unhelpful responses.

On The Effects Of Inequality On Economic Growth by Artir (Nintil) - Most of the article tries to explain and analyze the economic consensus on whether inequality harms growth. A very large number of papers are cited and discussed. A conclusion that the effect is at most small.


Two Kinds Of Caution by Scott Alexander - Sometimes boring technologies (ex container ships) wind up being far more important than flashy tech. However Scott argues that often the flashy tech really is important. There is too much contrarianism and not enough meta-contrarianism. AI risk.

Open Road by Scott Alexander - Bi-weekly public open thread. Some messages from Scott Alexander.

To The Great City by Scott Alexander - Scott's Karass is in San Fransisco. He is going home.

Open Thread 78 75 by Scott Alexander - Bi-weekly public open thread.

Why Are Transgender People Immune To Optical Illusions by Scott Alexander - Scott's community survey showed, with a huge effect size, that transgender individuals are less susceptible to the spinning mask and dancer illusions. Trans suffer from dis-associative disorder at a high rate. Connections between the two phenomena and NDMA. Commentary on the study methodology.

Contra Otium On Individualism by Scott Alexander (Scratchpad) - Eight point summary of Sarah's defense of individualism. Scott is terrified the market place of ideals doesn't work and his own values aren't memetically fit.

Conversation Deliberately Skirts The Border Of Incomprehensibility by Scott Alexander - Communication is often designed to be confusing so as to preserve plausible deniability.


Rethinking Reality And Rationality by mindlevelup - Productivity is almost a solved problem. Much current rationalist research is very esoteric. Finally grokking effective altruism. Getting people good enough at rationality that they are self correcting. Pedagogy and making research fields legible.

The Power Of Pettiness by Sarah Perry (ribbonfarm) - "These emotions – pettiness and shame – are the engines driving epistemic progress" Four virtues: Loneliness, ignorance, pettiness and overconfidence.

Irrationality is in the Eye of the Beholder by João Eira (Lettuce be Cereal) - Is eating a chocolate croissant on a diet always irrational? Context, hidden motivations and the curse of knowledge.

The Abyss Of Want by AellaGirl - The infinite regress of 'Asking why'. Taking acid and ego death. You can't imagine the experience of death. Coming back to life. Wanting to want things. Humility and fake enlightenment.

Epistemic Laws Of Motion by SquirrelInHell - Newton's three laws re-interpreted in terms of psychology and people's strategies. A worked example using 'physics' to determine if someone will change their mind. Short and clever.

Against Lone Wolf Selfimprovement by cousin_it (lesswrong) - Lone wolf improvement is hard. Too many rationalists attempt it for cultural and historical reasons. Its often better to take a class or find a group.

Fictional Body Language by Eukaryote - Body language in literature is often very extreme compared to real life. Emojis don't easily map to irl body language. A 'random' sample of how emotion in represented in American Gods, Earth and Lirael. Three strategies: Explicitly describing feelings vs describing actions vs metaphors.

Bayesian Probability Theory As Extended Logic A by ksvanhorn (lesswrong) - Cox's theorem is often cited to support that Bayesian probability is the only valid fundamental method of plausible reasoning. A simplified guide to Cox's theorem. The author their paper that uses weaker assumptions than Cox's theorem. The author's full paper and a more detailed exposition of Cox's theorem are linked.

Steelmanning The Chinese Room Argument by cousin_it (lesswrong) - A short thought experiment about consciousness and inferring knowledge from behavior.

Ideas On A Spectrum by Elo (BearLamp) - Putting ideas like 'selfishness' on a spectrum. Putting yourself and others on the spectrum. People who give you advice might disagree with you about where you fall on the spectrum. Where do you actually stand?

A Post Em Era Hint by Robin Hanson - In past ages there were pairs of innovations that enabled the emulation age without changing the growth rate. Forager: Reasoning and language. Farmer: Writing and math. Industrial: Computers and Digital Communication. What will the em-age equivalents be?

Zen Koans by Elo (BearLamp) - Connections between koans and rationalist ideas. A large number of koans are included at the end of the post. Audio of the associated meetup is included.

Fermi Paradox Resolved by Tyler Cowen - Link to a presentation. Don't just multiply point estimates. Which Drake parameters are uncertain. The Great filter is probably in the past. Lots of interesting graphs and statistics. Social norms and laws. Religion. Eusocial society.

Developmental Psychology In The Age Of Ems by Gordan (Map and Territory) - Brief intro to the Age of Em. Farmer values. Robin's approach to futurism. Psychological implications of most ems being middle aged. Em conservatism and maturity.

Call To Action by Elo (BearLamp) - Culmination of a 21 article series on life improvement and getting things done. A review of the series as a whole and thoughts on moving forward.

Cfar Week 1 by mindlevelup - What is working at CFAF actually like. Less rationality research than anticipated. Communication costs scale quadratically. Organization efficiency and group rationality.

Onemagisterium Bayes by tristanm (lesswrong) - Toolbox-ism is the dominant mode of thinking today. Downsides of toolbox-ism. Desiderata that imply Bayesianism. Major problems: Assigning priors and encountering new hypothesis. Four minor problems. Why the author is still a strong Bayesianism. Strong Bayesians can still use frequentist tools. AI Risk.

Selfconscious Ideology by casebash (lesswrong) - Lesswrong has a self conscious ideology. Self conscious ideologies have major advantages even if any given self-conscious ideology is flawed.

Intellectuals As Artists by Robin Hanson - Many norms function to show off individual impressiveness: Conversations, modern songs, taking positions on diverse subjects. Much intellectualism is not optimized for status gains not finding truth.

Just Saying What You Mean Is Impossible by Zvi Moshowitz - "Humans are automatically doing lightning fast implicit probabilistic analysis on social information in the background of every moment of their lives." This implies there is no way to divorce the content of your communication from its myriad probabilistic social implications. Different phrasings will just send different implications.

In Defense Of Individualist Culture by Sarah Constantin (Otium) - A description of individualist culture. Criticisms of individualist culture: Lacking sympathy, few good defaults. Defenses: Its very hard to change people (psychology research review). A defense of naive personal identity. Traditional culture is fragile. Building a community project is hard in the modern world, prepare for the failure modes. Modernity has big upsides, some people will make better choices than the traditional rules allow.

Forget The Maine by Robin Hanson - Monuments are not optimized for reminding people to do better. Instead they largely serve as vehicles for simplistic ideology.

The Ladder Of Interventions by mindlevelup - "This is a hierarchy of techniques to use for in-the-moment situations where you need to “convince” yourself to do something." The author uses these methods in practice.

On Dragon Army by Zvi Moshowitz - Long response to many quotes from "Dragon Army Barracks". Duncan't attitude to criticism. Tyler Durden shouldn't appeal to Duncan. Authoritarian group houses haven't been tried. Rationalists undervalue exploration. Loneliness and doing big things. The pendulum model of social progress. Sticking to commitments even when its painful. Saving face when you screw up. True Reliability: The bay is way too unreliable but Duncan goes too far. Trust and power Dynamics. Pragmatic criticism of the charter.


Updates To The Research Team And A Major Donation by The MIRI Blog - MIRIr received a 1 million dollar donation. Two new full-time researchers. Two researchers leaving. Medium term financial plans.

Conversation With Dario Amodei by Jeff Kaufman - "The research that's most valuable from an AI safety perspective also has substantial value from the perspective of solving problems today". Prioritize work on goals. Transparency and adversarial examples are also important.

Why Don't Ai Researchers Panic by Bayesian Investor - AI researchers predict a 5% chance of "extremely bad" (extinction level) events, why aren't they panicking? Answers: They are thinking of less bad worst cases, optimism about counter-measures, risks will be easy to deal with later, three "star theories" (MIRI, Paul Christiano, GOFAI). More answers: Fatal pessimism and resignation. It would be weird to openly worry. Benefits of AI-safety measures are less than the costs. Risks are distant.

Strategic Implications Of Ai Scenarios by (EA forum) - Questions and topics: Advanced AI timelines. Hard or soft takeoff? Goal alignment? Will advanced AI act as a single entity or a distributed system? Implication for estimating the EV of donating to AI-safety. - Tobias Baumann

Tool Use Intelligence Conversation by The Foundational Research Institute - A dialogue. Comparisons between humans and chimps/lions. The value of intelligence depends on the available tools. Defining intelligence. An addendum on "general intelligence" and factors that make intelligence useful.

Self-modification As A Game Theory Problem by (lesswrong) - "If I'm right, then any good theory for cooperation between AIs will also double as a theory of stable self-modification for a single AI, and vice versa." An article with mathematical details is linked.

Looking Into Ai Risk by Jeff Kaufman - Jeff is trying to decide if AI risk is a serious concern and whether he should consider working in the field. Jeff's AI-risk reading list. A large comment section with interesting arguments.


Ea Marketing And A Plea For Moral Inclusivity by MichaelPlant (EA forum) - EA markets itself as being about poverty reduction. Many EAs think other topics are more important (far future, AI, animal welfare, etc). The author suggests becoming both more inclusive and more openly honest.

My Current Thoughts On Miris Highly Reliable by Daniel Dewey (EA forum) - Report by the Open Phil AI safety lead. A basic description of and case for the MIRI program. Conclusion: 10% credence in MIRI's work being highly useful. Reasons: Hard to apply to early agents, few researchers are excited, other approaches seem more promising.

How Can We Best Coordinate As A Community by Benn Todd (EA forum) - 'Replaceability' is a bad reason not to do direct work, lots of positions are very hard to fill. Comparative Advantage and division of labor. Concrete ways to boost productivity: 5 minute favours, Operations roles, Community infrastructure, Sharing knowledge and Specialization. EA Global Video is included.

Deciding Whether to Recommend Fistula Management Charities by The GiveWell Blog - "An obstetric fistula, or gynecologic fistula, is an abnormal opening between the vagina and the bladder or rectum." Fistula management, including surgery. Open questions and uncertainty particularly around costs. Our plans to partner with IDinsight to answer these questions.

Allocating the Capital by GiveDirectly - Eight media links on Give Directly, Basic Income and Cash Transfers.

Testing An Ea Networkbuilding Strategy by remmelt (EA forum) - Pivot from supporting EA charities to cooperating with EA networks. Detailed goals, strategy, assumptions, metrics, collaborators and example actions.

How Long Does It Take To Research And Develop A Vaccine by (EA forum) - How long it takes to make a vaccine. Literature review. Historical data on how long a large number of vaccines took to develop. Conclusions.

Hi Im Luke Muehlhauser Ama About Open by Luke Muelhauser (EA forum) - Animal and computer consciousness. Luke wrote a report for the open philanthropy project on consciousness. Lots of high quality questions have been posted.

Hidden Cost Digital Convenience by Innovations for Poverty - Moving from in person to digital micro-finance can harm saving rates in developing countries. Reduction in group cohesion and visible transaction fees. Linked paper with details.

Projects People And Processes by Open Philosophy - Three approaches used by donors and decision makers: Choose from projects presented by experts, defer near-fully to trusted individuals and establishing systematic criteria. Pros and cons of each. Open Phil's current approach.

Effective Altruism An Idea Repository by Onemorenickname (lesswrong) - Effective altruism is less of a closed organization than the author thought. Building a better platform for effective altruist idea sharing.

Effective Altruism As Costly Signaling by Raemon (EA forum) - " 'a bunch of people saying that rich people should donate to X' is a less credible signal than 'a bunch of people saying X thing is important enough that they are willing to donate to it themselves.' "

The Person Affecting Philanthropists Paradox by MichaelPlant (EA forum) - Population ethics. The value of creating more happy people as opposed to making pre-existing people happy. Application to the question of whether to donate now or invest and donate later.

Oops Prize by Ben Hoffman (Compass Rose) - Positive norms around admitting you were wrong. Charity Science publicly admitted they were wrong about grant writing. Did anyone organization at EA Global admit they made a costly mistake? 1K oops prize.

===Politics and Economics:

Scraps 3 Hoffer And Performance Art by Lou (sam[]zdat) - Growing out of radicalism. Either economic and family instability can cause mass movements. why the left has adopted Freud. The Left's economic platform is popular, its cultural platform is not. Performance art: Marina Abramović's' 'Rhythm 0'. Recognizing and denying your own power.

What Replaces Rights And Discourse by Feddie deBoer - Lots of current leftist discourse is dismissive of rights and open discussion. But what alternative is there? The Soviets had bad justifications and a terrible system but at least it had an explicit philosophical alternative.

Why Do You Hate Elua by H i v e w i r e d - Scott's Elua as an Eldritch Abomination that threatens traditional culture. An extended sci-fi quote about Ra the great computer. "The forces of traditional values remembered an important fact: once you have access to the hardware, it’s over."

Why Did Europe Lose Crusades by Noah Smith - Technological comparison between Europe and the Middle East. Political divisions on both sides. Geographic distance. Lack of motivation.

Econtalk On Generic Medications by Aceso Under Glass - A few egregious ways that big pharma games the patent system. Short.

Data On Campus Free Speech Cases by Ozy (Thing of Things) - Ozy classifies a sample of the cases handled by the Foundation for Individual Rights in Education. Ozy classifies 77 cases as conservative, liberal or apolitical censorship. Conservative ideas were censored 52%, liberal 26% and apolitical 22%.

Beware The Moral Spotlight by Robin Hanson - The stated goals of government/business don't much matter compared to the selective pressures on their leadership, don't obsess over which sex has the worse deal overall, don't overate the benefits of democracy and ignore higher impact changes to government.

Reply To Yudkowsky by Bryan Caplan - Caplan quotes and replies to many sections Yudkowsky's response. Points: Yudkowsky's theory is a special case of Caplan's. The left has myriad complaints about markets. Empirically the market actually has consistently provided large benefits in many countries and times.

Without Belief In A God But Never Without Belief In A Devil by Lou (sam[]zdat) - The nature of mass movements. The beats and the John Birchers. The taxonomy of the frustrated. Horseshoe theory. The frustrated cannot derive satisfaction from action, something else has to fill the void Poverty, work and meaning. Mass movements need to sow resentment. Hatred is the strongest unifier. Modernity inevitably causes justified resentment. Tocqueville, Polyanai, Hoffer and Scott's theories. Helpful and unhelpful responses.

Genetic Behaviorism Supports The Influence Of Chance On Life Outcomes by Freddie deBoer - Much of the variance in many traits is non-shared-environment. Much non-shared-environment can be thought of as luck. In addition no one chooses or deserves their genes.

Yudkowsky On My Simpistic Theory of Left and Right by Bryan Caplan - Yudkowsky claims the left holds the market to the same standards as human beings. The market as a ritual holding back a dangerous Alien God. Caplan doesn't respond he just quotes Yudkowsky.

On The Effects Of Inequality On Economic Growth by Artir (Nintil) - Most of the article tries to explain and analyze the economic consensus on whether inequality harms growth. A very large number of papers are cited and discussed. A conclusion that the effect is at most small.


Erisology Of Self And Will Representative Campbell Speaks by Everything Studies - An exposition of the "mainstream" view of the self and free will.

What Is The Ein Sof The Meaning Of Perfection In by arisen (lesswrong) - "Kabbalah is based on the analogy of the soul as a cup and G-d as the light that fills the cup. Ein Sof, nothing ("Ein") can be grasped ("Sof"-limitation)."

Sexualtaboos by AellaGirl - A graph of sexual fetishes. The axes are "taboo-ness" and "reported interest". Taboo correlated negatively with interest (p < 0.01). Lots of fetishes are included and the sample size is pretty large.

Huffman Codes Problem by protokol2020 - Find the possible Huffman Codes for all twenty-six English letters.

If You're In School Try The Curriculum by Freddie deBoer - Ironic detachment "leaves you with the burden of the work but without the emotional support of genuine resolve". Don't be the sort of person who tweets hundreds of thousands of times but pretends they don't care about online.

Media Recommendations by Sailor Vulcan (BYS) - Various Reviews including: Games, Animated TV shows, Rationalist Pokemon. A more detailed review of Harry Potter and the Methods of Rationality.

Sunday Assorted Links by Tyler Cowen - Variety of Topics. Ethereum Cryptocurrency, NYC Diner decline, Building Chinese Airports, Soccer Images, Drone Wars, Harberger Taxation, Douthat on Heathcare.

Summary Of Reading April June 2017 by Eli Bendersky - Brief reviews. Various topics: Heavy on Economics. Some politics, literature and other topics.

Rescuing The Extropy Magazine Archives by deku_shrub (lesswrong) - "You'll find some really interesting very early articles on neural augmentation, transhumanism, libertarianism, AI (featuring Eliezer), radical economics (featuring Robin Hanson of course) and even decentralized payment systems."

Epistemic Spot Check A Guide To Better Movement Todd Hargrove by Aceso Under Glass - Flexibility and Chronic Pain. Early section on flexibility fails check badly. Section on psychosomatic pain does much better. Model: Simplicity (Good), Explanation (Fantastic), Explicit Predictions (Good), Useful Predictions (Poor), Acknowledge Limits (Poor), Measurability (Poor).

Book Review Barriers by Eukaryote - Even cell culturing is surprisingly hard if you don't know the details. There is not much institutional knowledge left in the field of bioweapons. Forcing labs underground makes bioterrorism even harder. However synthetic biology might make things much more dangerous.

Physics Problem 2 by protokol2020 - Can tidal forces rotate a metal wheel?

Poems by Scott Alexander (Scratchpad) - Violets aren't blue.

Evaluating Employers As Junior Software by Particular Virtue - You need to write alot of code and get detailed feedback to improve as an engineer. Practical suggestions to ensure your first job fulfills both conditions.


Kyle Maynard Without Limits by Tim Ferriss - "Kyle Maynard is a motivational speaker, bestselling author, entrepreneur, and ESPY award-winning mixed martial arts athlete, known for becoming the first quadruple amputee to reach the summit of Mount Kilimanjaro and Mount Aconcagua without the aid of prosthetics."

85 Is This The End Of Europe by Waking Up with Sam Harris - Douglas Murray and his book 'The Strange Death of Europe: Immigration, Identity, Islam'.

Myers Briggs, Diet, Mistakes And Immortality by Tim Ferriss - Ask me anything podcast. Topics beyond the title: Questions to prompt introspection, being a Jack of All Trades, balancing future and present goals, don't follow your passion, 80/20 memory retention, advice to your past selves.

Interview Ro Khanna Regional Development by Tyler Cowen - Bloomberg Podcast. "Technology, jobs and economic lessons from his perspective as Silicon Valley’s congressman."

Avic Roy by The Ezra Klein Show - Better Care Reconciliation Act, broader health care philosophies that fracture the right. Roy’s disagreements with the CBO’s methodology. The many ways he thinks the Senate bill needs to improve. How the GOP has moved left on health care policy. Medicaid, welfare reform, and the needy who are hard to help. The American health care system subsidizes the rich, etc.

Chris Blattman 2 by EconTalk - "Whether it's better to give poor Africans cash or chickens and the role of experiments in helping us figure out the answer. Along the way he discusses the importance of growth vs. smaller interventions and the state of development economics."

Landscapes Of Mind by Waking Up with Sam Harris - "why it’s so hard to predict future technology, the nature of intelligence, the 'singularity', artificial consciousness."

Blake Mycoskie by Tim Ferriss - Early entrepreneurial ventures. The power of journaling. How “the stool analogy” changed Blake’s life. Lessons from Ben Franklin.

Ben Sasse by Tyler Cowen - "Kansas vs. Nebraska, famous Nebraskans, Chaucer and Luther, unicameral legislatures, the decline of small towns, Ben’s prize-winning Yale Ph.d thesis on the origins of conservatism, what he learned as a university president, Stephen Curry, Chevy Chase, Margaret Chase Smith"

Danah Boyd on why Fake News is so Easy to Believe by The Ezra Klein Show - Fake news, digital white flight, how an anthropologist studies social media, machine learning algorithms reflect our prejudices rather than fixing them, what Netflix initially got wrong about their recommendations engine, the value of pretending your audience is only six people, the early utopian visions of the internet.

Robin Feldman by EconTalk - Ways pharmaceutical companies fight generics.

Jason Weeden On Do People Vote Based On Self Interest by Rational Speaking - Do people vote based on personality, their upbringing, blind loyalty or do they follow their self interest? What does self-interest even mean?

Reid Hoffman 2 by Tim Ferriss - The 10 Commandments of Startup Success according to the extremely successful investor Reid Hoffman.

[Link] Dissolving the Fermi Paradox (Applied Bayesianism)

11 shin_getter 03 July 2017 09:44AM

Self-modification as a game theory problem

11 cousin_it 26 June 2017 08:47PM

In this post I'll try to show a surprising link between two research topics on LW: game-theoretic cooperation between AIs (quining, Loebian cooperation, modal combat, etc) and stable self-modification of AIs (tiling agents, Loebian obstacle, etc).

When you're trying to cooperate with another AI, you need to ensure that its action will fulfill your utility function. And when doing self-modification, you also need to ensure that the successor AI will fulfill your utility function. In both cases, naive utility maximization doesn't work, because you can't fully understand another agent that's as powerful and complex as you. That's a familiar difficulty in game theory, and in self-modification it's known as the Loebian obstacle (fully understandable successors become weaker and weaker).

In general, any AI will be faced with two kinds of situations. In "single player" situations, you're faced with a choice like eating chocolate or not, where you can figure out the outcome of each action. (Most situations covered by UDT are also "single player", involving identical copies of yourself.) Whereas in "multiplayer" situations your action gets combined with the actions of other agents to determine the outcome. Both cooperation and self-modification are "multiplayer" situations, and are hard for the same reason. When someone proposes a self-modification to you, you might as well evaluate it with the same code that you use for game theory contests.

If I'm right, then any good theory for cooperation between AIs will also double as a theory of stable self-modification for a single AI. That means neither problem can be much easier than the other, and in particular self-modification won't be a special case of utility maximization, as some people seem to hope. But on the plus side, we need to solve one problem instead of two, so creating FAI becomes a little bit easier.

The idea came to me while working on this mathy post on IAFF, which translates some game theory ideas into the self-modification world. For example, Loebian cooperation (from the game theory world) might lead to a solution for the Loebian obstacle (from the self-modification world) - two LW ideas with the same name that people didn't think to combine before!

[Link] Putanumonit: What statistical power means, and why I'm terrified about psychology

11 Jacobian 21 June 2017 06:29PM

Bi-Weekly Rational Feed

11 deluks917 10 June 2017 09:56PM

===Highly Recommended Articles:

Bring Up Genius by Viliam (lesswrong) - An "80/20" translation. Positive motivation. Extreme resistance from the Hungarian government and press. Polgar's five principles. Biting criticism of the school system. Learning in early childhood. Is Genius a gift or curse? Celebrity. Detailed plan for daily instruction. Importance of diversity. Why chess? Teach the chess with love, playfully. Emancipation of women. Polgar's happy family.

The Shouting Class by Noah Smith - The majority of comments come from a tiny minority of commentators. Social media is giving a bullhorn to the people who constantly complain. Negativity is contagious. The level of discord in society is getting genuinely dangerous. The French Revolution. The author criticizes shouters on the Left and Right.

How Givewell Uses Cost Effectiveness Analyses by The GiveWell Blog - GiveWell doesn't take its estimates literally, unless one charity is measured as 2-3x as cost-effective GiveWell is unsure if a difference exists. Cost-effective is however the most important factor in GiveWell's recommendations. GiveWell goes into detail about how it deals with great uncertainty and suboptimal data.

Mode Collapse And The Norm One Principle by tristanm (lesswrong) - Generative Adversarial Networks. Applying the lessons of Machine Learning to discourse. How to make progress when the critical side of discourse is very powerful. "My claim is that any contribution to a discussion should satisfy the "Norm One Principle." In other words, it should have a well-defined direction, and the quantity of change should be feasible to implement."

The Face Of The Ice by Sarah Constantin (Otium) - Mountaineering. Survival Mindset vs Sexual-Selection Mindset. War and the Wilderness. Technical Skill.

Bayes: A Kinda Sorta Masterpost by Nostalgebraist - A long and very well thought-out criticism of Bayesianism. Explanation of Bayesian methodology. Comparison with classical statistics. Arguments for Bayes. The problem of ignored hypotheses with known relations. The problem of new ideas. Where do priors come from? Regularization and insights from machine learning.


SSC Journal Club Ai Timelines by Scott Alexander - A new paper surveying what Ai experts think about Ai progress. Contradictory results about when Ai will surpass humans at all tasks. Opinions on Ai risk, experts are taking the arguments seriously.

Terrorism and Involuntary Commitment by Scott Alexander (Scratchpad) - The leader of the terrorist attack in London was in a documentary about jihadists living in Britain. “Being the sort of person who seems likely to commit a crime isn’t illegal.” Involuntary commitment.

Is Pharma Research Worse Than Chance by Scott Alexander - The most promising drugs of the 21st century are MDMA and ketamine (third is psilocybin). These drugs were all found by the drug community. Maybe pharma should look for compounds with large effect sizes instead of searching for drugs with no side-effects.

Open Thread 77- Opium Thread by Scott Alexander - Bi-weekly open thread. Includes some comments of the week and an update on translating "Bringing Up Genius".

Third and Fourth Thoughts on Dragon Army by SlateStarScratchpad. - Scott goes from Anti-Anti-Dragon-Army to Anti-Dragon-Army. He then gets an email from Duncan and updates in favor of the position that Duncan thought things out well.

Hungarian Education III Mastering The Core Teachings Of The Budapestians by Scott Alexander - Lazlo Polgar wanted to prove he could intentionally raise chess geniuses. He raised the number 1,2 and 6 female chess players in the world?

Four Nobel Truths by Scott Alexander - Four Graphs describing facts about Israeli/Askenazi Nobel Prizes.


The Precept Of Niceness by H i v e w i r e d - Prisoner's Dilemma's. Even against a truly alien opponent you should still cooperate as long as possible on the iterated prisoner's dilemma, even with fixed round lengths, play tit-for-tat. Niceness is the best strategy.

Epistemology Vs Critical Thinking by Onemorenickname (lesswrong) - Epistemies work. General approaches don't work. Scientific approaches work. Epistemic effort vs Epistemic status. Criticisms of lesswrong Bayesianism.

Tasting Godhood by Agenty Duck - Poetic and personal. Wine tasting. Empathizing with other people. Seeing others as whole people. How to dream about other people. Sci-fi futures. Tasting godhood is the same as tasting other people. Looking for your own godhood.

Bayes: A Kinda Sorta Masterpost by Nostalgebraist - A long and very well thought-out criticism of Bayesianism. Explanation of Bayesian methodology. Comparison with classical statistics. Arguments for Bayes. The problem of ignored hypotheses with known relations. The problem of new ideas. Where do priors come from? Regularization and insights from machine learning.

Dichotomies by mindlevelup - 6 short essays about dichotomies and whats useful about noticing them. Fast vs Slow thinking. Focused vs Diffuse Mode. Clean vs Dirty Thinking. Inside vs Outside View. Object vs Meta level. Generative vs Iterative Mode. Some conclusions about the method.

How Men And Women Perceive Relationships Differently by AellaGirl - Survey Results about Relationship quality over time. Lots of graphs and a link to the raw data. "In summary, time is not kind. Relationships show an almost universal decrease in everything good the longer they go on. Poly is hard, and you have to go all the way to make it work – especially for men. Religion is also great, if you’re a man. Women get more excited and insecure, men feel undesirable."

Summer Programming by Jacob Falkovich (Put A Number On It!) - Jacob's Summer writing plan. Re-writing part of the lesswrong sequences. Ribbonfarm's longform blogging course on refactored perception.

Bet Or Update Fixing The Will to Wager Assumption by cousin_it (lesswrong) - Betting with better informed agents is irrational. Bayesian agents should however update their prior or agree to bet. Good discussion in comments.

Kindness Against The Grain by Sarah Constantin (Otium) - Sympathy and forgiveness evolved to follow local incentive gradients. Some details on we sympathize with and who we don't. The difference between a good deal and a sympathetic deal. Smooth emotional gradients and understanding what other people want. Forgiveness as not following the local gradient and why this can be useful.

Bring Up Genius by Viliam (lesswrong) - An "80/20" translation. Positive motivation. Extreme resistance from the Hungarian government and press. Polgar's five principles. Biting criticism of the school system. Learning in early childhood. Is Genius a gift or curse? Celebrity. Detailed plan for daily instruction. Importance of diversity. Why chess? Teach the chess with love, playfully. Emancipation of women. Polgar's happy family.

Deorbiting A Metaphor by H i v e w i r e d - Another post in the origin sequence. Rationalist Myth-making. (note: I am unlikely to keep linking all of these. Follow hivewired’s blog)

Conformity Excuses by Robin Hanson - Human behavior is often explained by pressure to conform. However we consciously experience much less pressure. Robin discusses a list of ways to rationalize conforming.

Becoming A Better Community by Sable (lesswrong) - Lesswrong holds its memebers to a high standard. Intimacy requires unguarded spontaneous interactions. Concrete ideas to add more fun and friendship to lesswrong.

Optimizing For Meta Optimization by H i v e w i r e d - A very long list of human cultural universals and comments on which ones to encourage/discourage: Myths, Language, Cognition, Society. Afterwards some detailed bullet points about an optimal dath ilanian culture.

On Resignation by Small Truths - Artificial intelligence. "It’s an embarrassing lapse, but I did not think much about how the very people who already know all the stuff I’m learning would behave. I wasn’t thinking enough steps ahead. Seen in this context, Neuralink isn’t an exciting new tech venture so much as a desperate hope to mitigate an unavoidable disaster."

Cognitive Sciencepsychology As A Neglected by Kaj Sotala (EA forum) - Ways psychology could benefit AI safety: "The psychology of developing an AI safety culture, Developing better analyses of 'AI takeoff' scenarios, Defining just what it is that human values are, Better understanding multi-level world-models." Lots of interesting links.

Mode Collapse And The Norm One Principle by tristanm (lesswrong) - Generative Adversarial Networks. Applying the lessons of Machine Learning to discourse. How to make progress when the critical side of discourse is very powerful. "My claim is that any contribution to a discussion should satisfy the "Norm One Principle." In other words, it should have a well-defined direction, and the quantity of change should be feasible to implement."

Finite And Infinite by Sarah Constantin (Otium) - "James Carse, in Finite and Infinite Games, sets up a completely different polarity, between infinite game-playing (which is open-ended, playful, and non-competitive) vs. finite game-playing (which is definite, serious, and competitive)." Playfulness, property, and cooperating with people who seriously weird you out.

Script for the rationalist seder is linked by Raemon (lesswrong) - An explanation of Rationalist Seder, a remix of the Passover Seder refocused on liberation in general. A story of two tribes and the power of stories. The full Haggadah/script for the rationalist Seder is linked.

The Personal Growth Cycle by G Gordon Worley (Map and Territory) - Stages of Development. "Development starts from a place of integration, followed by disintegration into confusion, which through active efforts at reintegration in a safe space results in development. If a safe space for reintegration is not available, development may not proceed."

Until We Build Dath Ilan by H i v e w i r e d - Eliezer's Sci-fi utopia Dath Ilan. The nature of the rationalist community. A purpose for the rationality community. Lots of imagery and allusions. A singer is someone who tries to do good.

Do Ai Experts Exist by Bayesian Investor - Some of the numbers from " When Will AI Exceed Human Performance? Evidence from AI Experts" don't make sense.

Relinquishment Cultivation by Agenty Duck - Agenty Duck designs meditation to cultivate the attitude of "If X is true I wish to believe X, if X is not true I wish to believe not X". The technique is inspired by 'loving-kindness' meditation.

10 Incredible Weaknesses Of The Mental Health by arunbharatula (lesswrong) - Ten arguments that undermine the credibility of the mental health workforce. Some of the arguments are sourced and argued significantly more thoroughly than other.

Philosophical Parenthood by SquirrelInHell - Updateless Decision theory. Ashkenazi intelligence. "In this post, I will lay out a strong philosophical argument for rational and intelligent people to have children. It's important and not obvious, so listen well."

On Connections Between Brains And Computers by Small Truths - A condensation of Tim Ubran's 36K word article about Neuralink. The astounding benefits of having even a SIRI level Ai responding directly to your thoughts. The existential threat of Ai means that mind-computer links are worth the risks.

Thoughts Concerning Homeschooling by Ozy (Thing of Things) - Evidence that many public school practices are counter-productive. Stats on the academic performance of home-schoolers. Educating 'weird awkward nerds'.

The Face Of The Ice by Sarah Constantin (Otium) - Mountaineering. Survival Mindset vs Sexual-Selection Mindset. War and the Wilderness. Technical Skill.


Review Of Ea New Zealands Doing Good Better Book by cafelow (EA forum) - New Zealand EAs gave out 250 copies of "Doing Good Better". 80 of the recipients responded to a follow up survey. The results were extremely encouraging. Survey details and discussion. Possible flaws with the giveaway and survey.

Announcing Effective Altruism Grants by Maxdalton (EA forum) - CEA is giving out £100,000 grants for personal projects. "We believe that providing those people with the resources that they need to realize their potential could be a highly effective use of resources." A list of what projects could get funded, the list is very broad. Evaluation criteria.

A Powerful Weapon in the Arsenal (Links Post) by GiveDirectly - 8 Links on Basic Income, Effective Altruism, Cash Transfers and Donor Advised Funds

A Paradox In The Measurement Of The Value Of Life by klloyd (EA forum) - Eight Thousand words on: “A Health Economics Puzzle: Why are there apparent inconsistencies in the monetary valuation of a statistical life (VSL) and a quality-adjusted life year (QALY$)?”

New Report Consciousness And Moral Patienthood by Open Philosophy - “In short, my tentative conclusions are that I think mammals, birds, and fishes are more likely than not to be conscious, while (e.g.) insects are unlikely to be conscious. However, my probabilities are very “made-up” and difficult to justify, and it’s not clear to us what actions should be taken on the basis of such made-up probabilities.”

Adding New Funds To Ea Funds by the Center for Effective Altruism (EA forum) - The Center for Effective Altruism wants feedback on whether it should add more EA funds. Each question is followed by a detailed list of critical considerations.

How Givewell Uses Cost Effectiveness Analyses by The GiveWell Blog - GiveWell doesn't take its estimates literally, unless one charity is measured as 2-3x as cost-effective GiveWell is unsure if a difference exists. Cost-effective is however the most important factor in GiveWell's recommendations. GiveWell goes into detail about how it deals with great uncertainty and suboptimal data.

The Time Has come to Find Out [Links] by GiveDirectly - 8 media links related to Cash Transfers, Give Directly and Effective Altruism.

Considering Considerateness: Why Communities Of Do Gooders Should Be by The Center for Effective Altruism - Consequentialist reasons to be considerate and trustworthy. Detailed and contains several graphs. Include practical discussions of when not to be considerate and how to handle unreasonable preferences. The conclusion discusses how considerate EAs should be. The bibliography contains many very high quality articles written by the community.

===Politics and Economics:

Summing Up My Thoughts On Macroeconomics by Noah Smith - Slides from Noah's talk at the Norwegian Finance Ministry. Comparison of Industry, Central Bank and Academic Macroeconomics. Overview of important critiques of academic macro. The DGSE standard mode and ways to improve it. What makes a good Macro theory. Go back to the microfoundations.

Why Universities Cant Be The Primary Site Of Political Organizing by Freddie deBoer - Few people on campus. Campus activism is seasonal. Students are an itinerant population. Town and gown conflicts. Students are too busy. First priority is employment. Is activism a place for student growth?. Labor principles.

Some Observations On Cis By Default Identification by Ozy (Thing of Things) - Many 'cis-by-default' people are repressing or not noticing their gender feelings. This effect strongly depends on a person's community.

One Day We Will Make Offensive Jokes by AellaGirl - "This is why I feel suspicious of some groups that strongly oppose offensive jokes – they have the suspicion that every person is like my parents – that every human “actually wants” all the terrible things to happen."

Book Review Weapons Of Math Destruction by Zvi Moshowitz - Extremely long. "What the book is actually mostly about on its surface, alas, is how bad and unfair it is to be a Bayesian. There are two reasons, in her mind, why using algorithms to be a Bayesian is just awful."

A Brief Argument With Apparently Informed Global Warming Denialists by Artir (Nintil) - Details of the back and forth argument. So commentary on practical rationality and speculation about how the skeptic might have felt.

The Shouting Class by Noah Smith - The majority of comments come from a tiny minority of commentators. Social media is giving a bullhorn to the people who constantly complain. Negativity is contagious. The level of discord in society is getting genuinely dangerous. The French Revolution. The author criticizes shouters on the Left and Right.

Population By Country And Region 10K BCE to 2016 CE by Luke Muehlhauser - 204 countries, 27 region. Links to the database used and a forthcoming explanatory paper. From 10K BCE to 0 CE gaps are 1000 years. From 0 CE to 1700 CE gaps are 100 years. After that they are 10 years long.

Regulatory Lags For New Technology 2013 Notes by gwern (lesswrong) - Gwern looks at the history of regulation for high frequency trading, self driving cars and hacking. The post is mostly comprised of long quotes from articles linked by gwern.

Two Economists Ask Teachers To Behave As Irrational Actors by Freddie deBoer - A response to Cowen's interview of Raj Chetty. Standard Education reform rhetoric implies that hundreds of thousands of teachers need to be fired. However teachers don't control most of the important inputs to student performance. You won't get more talented teachers unless you increase compensation.

Company Revenue Per Employee by Tyler Cowen - The energy sector has high revenue per employee. The highest score was attained by a pharmaceutical distributor. Hotels, restaurants and consumer discretionaries do the worst on this metric. Tech has a middling performance.


A Remark On Usury by Entirely Useless - "To take usury for money lent is unjust in itself, because this is to sell what does not exist, and this evidently leads to inequality which is contrary to justice." Thomas Aquinas is quoted at length explaining the preceding statement. EntirelyUseless argues that Aquinas mixes up the buyer and the seller.

Bike To Work Houston by Mr. Money Mustache - How a lawyer bikes to work in Houston. Bikes are surprisingly fast relative to cars in cities. Houston is massive.

Fuckers Vs Raisers by AellaGirl - Evolutionary psychology. The qualities that are attractive in a guy who sleeps around are also attractive in a guy who wants to settle down.

Reducers Transducers And Coreasync In Clojure by Eli Bendersky - "I find it fascinating how one good idea (reducers) morphed into another (transducers), and ended up mating with yet another, apparently unrelated concept (concurrent pipelines) to produce some really powerful coding abstractions."

Thingness And Thereness by Venkatesh Rao (ribbonfarm) - The relation between politics, home and frontier. Big Data, deep learning and the blockchain. Liminal spaces and conditions.

Create 2314 by protokol2020 - Find the shortest algorithm to create the number 2314 using a prescribed set of operations.

Text To Speech Speed by Jeff Kaufman - Text to speech has become a very efficient way to interact with computers. Questions about settings. Very short.

Hello World! Stan, Pymc3 and Edward by Bob Carpenter (Gelman's Blog) - Comparison of the three frameworks. Test case of Bayesian linear regression. Extendability and efficiency of the frameworks is discussed.

Computer Science Majors by Tyler Cowen - Tyler links to an article by Dan wang. The author gives 11 reasons why CS majors are rare, none of which he finds convincing. Eventually the author seems to conclude that the 2001 bubble, changing nature of the CS field, power law distribution in developer productivity and lack of job security are important causes.

Beespotting On I-5 by Eukaryote - Drive from San Fran to Seattle. The vast agricultural importance of Bees. Improving Bee quality of life.


81 Leaving Islam by Waking Up with Sam Harris - "Sarah Haider. Her organization Ex-Muslims of North America, how the political Left is confused about Islam, "rape culture" under Islam, honesty without bigotry, stealth theocracy, immigration, the prospects of reforming Islam"

Newcomers by Venam - A transcript of a discussion about advice for new Unix users. Purpose. Communities. Learning by Yourself. Technical Tips. Venam linked tons of podcast transcripts today. Check them out.

Masha Gessen, Russian-American Journalist by The Ezra Klein Show - Trump and Russia, plausible and sinister explanation. Ways Trump is and isn't like Putin, studying autocracies, the psychology of Jared Kushner

Christy Ford by EconTalk - "A history of how America's health care system came to be dominated by insurance companies or government agencies paying doctors per procedure."

Nick Szabo by Tim Ferriss - "Computer scientist, legal scholar, and cryptographer best known for his pioneering research in digital contracts and cryptocurrency."

The Road To Tyranny by Waking Up with Sam Harris - Timothy Snyder. His book On Tyranny: Twenty Lessons from the Twentieth Century.

Hans Noel On The Role Of Ideology In Politics by Rational Speaking - "Why the Democrats became the party of liberalism and the Republicans the party of conservatism, whether voters are hypocrites in the way they apply their ostensible ideology, and whether politicians are motivated by ideals or just self-interest."

Prediction should be a sport

10 chaosmage 10 August 2017 07:55AM

So, I've been thinking about prediction markets and why they aren't really catching on as much as I think they should.

My suspicion is that (beside Robin Hanson's signaling explanation, and the amount of work it takes to get to the large numbers of predictors where the quality of results becomes interesting) the basic problem of prediction markets is that they look and feel like gambling. Or at best like the stock market, which for the vast majority of people is no less distasteful.

Only a small minority of people are neither disgusted by nor terrified of gambling. Prediction markets right now are restricted to this small minority.

Poker used to have the same problem.

But over the last few decades Poker players have established that Poker is (also) a sport. They kept repeating that winning isn't purely a matter of luck, they acquired the various trappings of tournaments and leagues, they developed a culture of admiration for the most skillful players that pays in prestige rather than only money and makes it customary for everyone involved to show their names and faces. For Poker, this has worked really well. There are much more Poker players, more really smart people are deciding to get into Poker and I assume the art of game probably improved as well.

So we should consider re-framing prediction the same way.

The calibration game already does this to a degree, but sport needs competition, so results need to be comparable, so everyone needs to make predictions on the same events. You'd need something like standard cards of events that players place their predictions on.

Here's a fantasy of what it could look like.

  • Late in the year, a prediction tournament starts with the publication of a list of events in the coming year. Everybody is invited to enter the tournament (and maybe pay a small participation fee) by the end of the year, for a chance to be among the best predictors and win fame and prizes.
  • Everyone who enters plays the calibration game on the same list of events. All predictions are made public as soon as the submission period is over and the new year begins. Lots of discussion of each event's distribution of predictions.
  • Over the course of the year, events on the list happen or fail to happen. This allows for continually updated scores, a leaderboard and lots of blogging/journalistic opportunities.
  • Near the end of the year, as the leaderboard turns into a shortlist of potential winners, tension mounts. Conveniently, this is also when the next tournament starts.
  • At new year's, the winner is crowned (and I'm open to having that happen literally) at a big celebration which is also the end of the submission period for the next tournament and the revelation of what everyone is predicting for this next round. This is a big event that happens to be on a holiday, where more people have time for big events.
Of course this isn't intended to replace existing prediction markets. It is an addition to those, a fun and social thing with lots of PR potential and many opportunities to promote rationality. It should attract people to prediction who are not attracted to prediction markets. And it could be prototyped pretty cheaply, and developed further if it is as much fun as I think it would be.

[Link] Daniel Dewey on MIRI's Highly Reliable Agent Design Work

10 lifelonglearner 09 July 2017 04:35AM

AI Summer Fellows Program

9 abramdemski 06 August 2017 07:35AM

CFAR is running a free two-week program this September, aimed at increasing participant's ability to do technical research in AI alignment. Like the MIRI Summer Fellows Program which ran the past two years, this will include CFAR course material, plus content on AI alignment research and time to collaborate on research with other participants and researchers such as myself! It will be located in the SF Bay area, September 8-25. See more information and apply here.

MILA gets a grant for AI safety research

9 Dr_Manhattan 21 July 2017 03:34PM

The really good news is that Yoshua Bengio is leading this (he is extremely credible in modern AI/deep learning world), and this is a pretty large change of mind for him. When I spoke to him at a conference 3 years ago he was pretty dismissive of the whole issue; this year's FLI conference seems to have changed his mind (kudos to them)

Of course huge props to OpenPhil for pursuing this

Bayesian probability theory as extended logic -- a new result

9 ksvanhorn 06 July 2017 07:14PM

I have a new paper that strengthens the case for strong Bayesianism, a.k.a. One Magisterium Bayes. The paper is entitled "From propositional logic to plausible reasoning: a uniqueness theorem." (The preceding link will be good for a few weeks, after which only the preprint version will be available for free. I couldn't come up with the $2500 that Elsevier makes you pay to make your paper open-access.)

Some background: E. T. Jaynes took the position that (Bayesian) probability theory is an extension of propositional logic to handle degrees of certainty -- and appealed to Cox's Theorem to argue that probability theory is the only viable such extension, "the unique consistent rules for conducting inference (i.e. plausible reasoning) of any kind." This position is sometimes called strong Bayesianism. In a nutshell, frequentist statistics is fine for reasoning about frequencies of repeated events, but that's a very narrow class of questions; most of the time when researchers appeal to statistics, they want to know what they can conclude with what degree of certainty, and that is an epistemic question for which Bayesian statistics is the right tool, according to Cox's Theorem.

You can find a "guided tour" of Cox's Theorem here (see "Constructing a logic of plausible inference"). Here's a very brief summary. We write A | X for "the reasonable credibility" (plausibility) of proposition A when X is known to be true. Here X represents whatever information we have available. We are not at this point assuming that A | X is any sort of probability. A system of plausible reasoning is a set of rules for evaluating A | X. Cox proposed a handful of intuitively-appealing, qualitative requirements for any system of plausible reasoning, and showed that these requirements imply that any such system is just probability theory in disguise. That is, there necessarily exists an order-preserving isomorphism between plausibilities and probabilities such that A | X, after mapping from plausibilities to probabilities, respects the laws of probability.

Here is one (simplified and not 100% accurate) version of the assumptions required to obtain Cox's result:


  1. A | X is a real number.
  2. (A | X) = (B | X) whenever A and B are logically equivalent; furthermore, (A | X) ≤ (B | X) if B is a tautology (an expression that is logically true, such as (a or not a)).
  3. We can obtain (not A | X) from A | X via some non-increasing function S. That is, (not A | X) = S(A | X).
  4. We can obtain (A and B | X) from (B | X) and (A | B and X) via some continuous function F that is strictly increasing in both arguments: (A and B | X) = F((A | B and X), B | X).
  5. The set of triples (x,y,z) such that x = A|X, y = (B | A and X), and z = (C | A and B and X) for some proposition A, proposition B, proposition C, and state of information X, is dense. Loosely speaking, this means that if you give me any (x',y',z') in the appropriate range, I can find an (x,y,z) of the above form that is arbitrarily close to (x',y',z').
The "guided tour" mentioned above gives detailed rationales for all of these requirements.

Not everyone agrees that these assumptions are reasonable. My paper proposes an alternative set of assumptions that are intended to be less disputable, as every one of them is simply a requirement that some property already true of propositional logic continue to be true in our extended logic for plausible reasoning. Here are the alternative requirements:
  1. If X and Y are logically equivalent, and A and B are logically equivalent assuming X, then (A | X) = (B | Y).
  2. We may define a new propositional symbol s without affecting the plausibility of any proposition that does not mention that symbol. Specifically, if s is a propositional symbol not appearing in A, X, or E, then (A | X) = (A | (s ↔ E) and X).
  3. Adding irrelevant background information does not alter plausibilities. Specifically, if Y is a satisfiable propositional formula that uses no propositional symbol occurring in A or X, then (A | X) = (A | Y and X).
  4. The implication ordering is preserved: if  A → B is a logical consequence of X, but B → A is not, then then A | X < B | X; that is, A is strictly less plausible than B, assuming X.
Note that I do not assume that A | X is a real number. Item 4 above assumes only that there is some partial ordering on plausibility values: in some cases we can say that one plausibility is greater than another.


I also explicitly take the state of information X to be a propositional formula: all the background knowledge to which we have access is expressed in the form of logical statements. So, for example, if your background information is that you are tossing a six-sided die, you could express this by letting s1 mean "the die comes up 1," s2 mean "the die comes up 2," and so on, and your background information X would be a logical formula stating that exactly one of s1, ..., s6 is true, that is,

(s1 or s2 or s3 or s5 or s6) and
not (s1 and s2) and not (s1 and s3) and not (s1 and s4) and
not (s1 and s5) and not (s1 and s6) and not (s2 and s3) and
not (s2 and s4) and not (s2 and s5) and not (s2 and s6) and
not (s3 and s4) and not (s3 and s5) and not (s3 and s6) and
not (s4 and s5) and not (s4 and s6) and not (s5 and s6).

Just like Cox, I then show that there is an order-preserving isomorphism between plausibilities and probabilities that respects the laws of probability.

My result goes further, however, in that it gives actual numeric values for the probabilities. Imagine creating a truth table containing one row for each possible combination of truth values assigned to each atomic proposition appearing in either A or X. Let n be the number of rows in this table for which X evaluates true. Let m be the number of rows in this table for which both A and X evaluate true. If P is the function that maps plausibilities to probabilities, then P(A | X) = m / n.

For example, suppose that a and b are atomic propositions (not decomposable in terms of more primitive propositions), and suppose that we only know that at least one of them is true; what then is the probability that a is true? Start by enumerating all possible combinations of truth values for a and b:
  1. a false, b false: (a or b) is false, a is false.
  2. a false, b true : (a or b) is true,  a is false.
  3. a true,  b false: (a or b) is true,  a is true.
  4. a true,  b true : (a or b) is true,  a is true.
There are 3 cases (2, 3, and 4) in which (a or b) is true, and in 2 of these cases (3 and 4) a is also true. Therefore,

    P(a | a or b) = 2/3.

This concords with the classical definition of probability, which Laplace expressed as

The probability of an event is the ratio of the number of cases favorable to it, to the number of possible cases, when there is nothing to make us believe that one case should occur rather than any other, so that these cases are, for us, equally possible.

This definition fell out of favor, in part because of its apparent circularity. My result validates the classical definition and sharpens it. We can now say that a “possible case” is simply a truth assignment satisfying the premise X. We can simply drop the problematic phrase “these cases are, for us, equally possible.” The phrase “there is nothing to make us believe that one case should occur rather than any other” means that we possess no additional information that, if added to X, would expand by differing multiplicities the rows of the truth table for which X evaluates true.

For more details, see the paper linked above.

One-Magisterium Bayes

9 tristanm 29 June 2017 11:02PM

[Epistemic Status: Very partisan / opinionated. Kinda long, kinda rambling.]

In my conversations with members of the rationalist community as well as in my readings of various articles and blog posts produced by them (as well as outside), I’ve noticed a recent trend towards skepticism of Bayesian principles and philosophy (see Nostalgebraist’s recent post for an example), which I have regarded with both surprise and a little bit of dismay, because I think progress within a community tends to be indicated by moving forward to new subjects and problems rather than a return to old ones that have already been extensively argued for and discussed. So the intent of this post is to summarize a few of the claims I’ve seen being put forward and try to point out where I believe these have gone wrong.

It’s also somewhat an odd direction for discussion to be going in, because the academic statistics community has largely moved on from debates between Bayesian and Frequentist theory, and has largely come to accept both the Bayesian and the Frequentist / Fisherian viewpoints as valid. When E.T. Jaynes wrote his famous book, the debate was mostly still raging on, and many questions had yet to be answered. In the 21st century, statisticians have mostly come to accept a world in which both approaches exist and have their merits.

Because I will be defending the Bayesian side here, there is a risk that this post will come off as being dogmatic. We are a community devoted to free-thought after all, and any argument towards a form of orthodoxy might be perceived as an attempt to stifle dissenting viewpoints. That is not my intent here, and in fact I plan on arguing against Bayesian dogmatism as well. My goal is to argue that having a base framework with which to feel relatively high confidence in is useful to the goals of the community, and that if we feel high enough confidence in it, then spending  extra effort trying to prove it false might be wasting brainpower than can potentially be used on more interesting or useful tasks. There could always be a point we reach where most of us strongly feel that unless we abandon Bayesianism, we can’t make any further progress. I highly doubt that we have reached such a point or that we ever will.

This is also a personal exercise to test my understanding of Bayesian theory and my ability to communicate it. My hope is that if my ideas here are well presented, it should be much easier for both myself and others to find flaws with it and allow me to update.

I will start with an outline of philosophical Bayesianism, also called “Strong Bayesianism”, or what I prefer to call it, “One Magisterium Bayes.” The reason for wanting to refer to it as being a single magisterium will hopefully become clear. The Sequences did argue for this point of view, however, I think the strength of the Sequences had more to do with why you should update your beliefs in the face of new evidence, rather than why Bayes' theorem was the correct way to do this. In contrast, I think the argument for using Bayesian principles as the correct set of reasoning principles was made more strongly by E.T. Jaynes. Unfortunately, I feel like his exposition of the subject tends to get ignored relative to the material presented in the Sequences. Not that the information in the Sequences isn’t highly relevant and important, just that Jaynes' arguments are much more technical, and their strength can be overlooked for this reason. 

The way to start an exposition on one-magisterium rationality is by contrast to multi-magisteria modes of thought. I would go as far as to argue that the multi-magisterium view, or what I sometimes prefer to call tool-boxism, is by far the most dominant way of thinking today. Tool-boxism can be summarized by “There is no one correct way to arrive at the truth. Every model we have today about how to arrive at the correct answer is just that – a model. And there are many, many models. The only way to get better at finding the correct answer is through experience and wisdom, with a lot of insight and luck, just as one would master a trade such as woodworking. There’s nothing that can replace or supersede the magic of human creativity. [Sometimes it will be added:] Also, don’t forget that the models you have about the world are heavily, if not completely, determined by your culture and upbringing, and there’s no reason to favor your culture over anyone else’s.”

As I hope to argue in this post, tool-boxism has many downsides that should push us further towards accepting the one-magisterium view. It also very dramatically differs in how it suggests we should approach the problem of intelligence and cognition, with many corollaries in both rationalism and artificial intelligence. Some of these corollaries are the following:

  • If there is no unified theory of intelligence, we are led towards the view that recursive self-improvement is not possible, since an increase in one type of intelligence does not necessarily lead to an improvement in a different type of intelligence.
  • With a diversification in different notions of correct reasoning within different domains, it heavily limits what can be done to reach agreement on different topics. In the end we are often forced to agree to disagree, which while preserving social cohesion in different contexts, can be quite unsatisfying from a philosophical standpoint.
  • Related to the previous corollary, it may lead to beliefs that are sacred, untouchable, or based on intuition, feeling, or difficult to articulate concepts. This produces a complex web of topics that have to be avoided or tread carefully around, or a heavy emphasis on difficult to articulate reasons for preferring one view over the other.
  • Developing AI around a tool-box / multi-magisteria approach, where systems are made up of a wide array of various components, limits generalizability and leads to brittleness. 

One very specific trend I’ve noticed lately in articles that aim to discredit the AGI intelligence explosion hypothesis, is that they tend to take the tool-box approach when discussing intelligence, and use that to argue that recursive self-improvement is likely impossible. So rationalists should be highly interested in this kind of reasoning. One of Eliezer’s primary motivations for writing the Sequences was to make the case for a unified approach to reasoning, because it lends credence to the view of intelligence in which intelligence can be replicated by machines, and where intelligence is potentially unbounded. And also that this was a subtle and tough enough subject that it required hundreds of blog posts to argue for it. So because of the subtle nature of the arguments I’m not particularly surprised by this drift, but I am concerned about it. I would prefer if we didn’t drift.

I’m trying not to sound No-True-Scotsman-y here, but I wonder what it is that could make one a rationalist if they take the tool-box perspective. After all, even if you have a multi-magisterium world-view, there still always is an underlying guiding principle directing the use of the proper tools. Often times, this guiding principle is based on intuition, which is a remarkably hard thing to pin down and describe well. I personally interpret the word ‘rationalism’ as meaning in the weakest and most general sense that there is an explanation for everything – so intelligence isn’t irreducibly based on hand-wavy concepts such as ingenuity and creativity. Rationalists believe that those things have explanations, and once we have those explanations, then there is no further use for tool-boxism.

I’ll repeat the distinction between tool-boxism and one-magisterium Bayes, because I believe it’s that important: Tool-boxism implies that there is no underlying theory that describes the mechanisms of intelligence. And this assumption basically implies that intelligence is either composed of irreducible components (where one component does not necessarily help you understand a different component) or some kind of essential property that cannot be replicated by algorithms or computation.

Why is tool-boxism the dominant paradigm then? Probably because it is the most pragmatically useful position to take in most circumstances when we don’t actually possess an underlying theory. But the fact that we sometimes don’t have an underlying theory or that the theory we do have isn’t developed to the point where it is empirically beating the tool box approach is sometimes taken as evidence that there isn't a unifying theory. This is, in my opinion, the incorrect conclusion to draw from these observations.

Nevertheless, it seems like a startlingly common conclusion to draw. I think the great mystery is why this is so. I don’t have very convincing answers to that question, but I suspect it has something to do with how heavily our priors are biased against a unified theory of reasoning. It may also be due to the subtlety and complexity of the arguments for a unified theory. For that reason, I highly recommend reviewing those arguments (and few people other than E.T. Jaynes and Yudkowsky have made them). So with that said, let’s review a few of those arguments, starting with one of the myths surrounding Bayes theorem I’d like to debunk:

Bayes Theorem is a trivial consequence of the Kolmogorov Axioms, and is therefore not powerful.

This claim usually presented as part of a claim that “Bayesian” probability is just a small part of regular probability theory, and therefore does not give us any more useful information than you’d get from just studying probability theory. And as a consequence of that, if you insist that you’re a “Strong” Bayesian, that means you’re insisting on using only on that small subset of probability theory and associated tools we call Bayesian.

And the part of the statement that says the theorem is a trivial consequence of the Kolmogorov axioms is technically true. It’s the implication typically drawn from this that is false. The reason it’s false has to do with Bayes theorem being a non-trivial consequence of a simpler set of axioms / desiderata. This consequence is usually formalized by Cox’s theorem, which is usually glossed over or not quite appreciated for how far-reaching it actually is.

Recall that the qualitative desiderata for a set of reasoning rules were:

  1. Degrees of plausibility are represented by real numbers.
  2. Qualitative correspondence with common sense.
  3. Consistency. 

You can read the first two chapters of Jaynes’ book, Probability Theory: The Logic of Science if you want more detail into what those desiderata mean. But the important thing to note from them is that they are merely desiderata, not axioms. This means we are not assuming those things are already true, we just want to devise a system that satisfies those properties. The beauty of Cox’s theorem is that it specifies exactly one set of rules that satisfy these properties, of which Bayes Theorem as well as the Kolmogorov Axioms are a consequence of those rules.

The other nice thing about this is that degrees of plausibility can be assigned to any proposition, or any statement that you could possibly assign a truth value to. It does not limit plausibility to “events” that take place in some kind of space of possible events like whether a coin flip comes up heads or tails. What’s typically considered the alternative to Bayesian reasoning is Classical probability, sometimes called Frequentist probability, which only deals with events drawn from a sample space, and is not able to provide methods for probabilistic inference of a set of hypotheses.

For axioms, Cox’s theorem merely requires you to accept Boolean algebra and Calculus to be true, and then you can derive probability theory as extended logic from that. So this should be mindblowing, right? One Magisterium Bayes? QED? Well apparently this set of arguments is not convincing to everyone, and it’s not because people find Boolean logic and calculus hard to accept.

Rather, there are two major and several somewhat minor difficulties encountered within the Bayesian paradigm. The two major ones are as follows:

  • The problem of hypothesis generation.
  • The problem of assigning priors. 

The list of minor problems are as follows, although like any list of minor issues, this is definitely not exhaustive:

  • Should you treat “evidence” for a hypothesis, or “data”, as having probability 1?
  • Bayesian methods are often computationally intractable.
  • How to update when you discover a “new” hypothesis.
  • Divergence in posterior beliefs for different individuals upon the acquisition of new data.

Most Bayesians typically never deny the existence of the first two problems. What some anti-Bayesians conclude from them, though, is that Bayesianism must be fatally flawed due to those problems, and that there is some other way of reasoning that would avoid or provide solutions to those problems. I’m skeptical about this, and the reason I’m skeptical is because if you really had a method for say, hypothesis generation, this would actually imply logical omniscience, and would basically allow us to create full AGI, RIGHT NOW. If you really had the ability to produce a finite list containing the correct hypothesis for any problem, the existence of the other hypotheses in this list is practically a moot point – you have some way of generating the CORRECT hypothesis in a finite, computable algorithm. And that would make you a God.

As far as I know, being able to do this would imply that P = NP is true, and as far as I know, most computer scientists do not think it’s likely to be true (And even if it were true, we might not get a constructive proof from it).  But I would ask: Is this really a strike against Bayesianism? Is the inability of Bayesian theory to provide a method for providing the correct hypothesis evidence that we can’t use it to analyze and update our own beliefs?

I would add that there are plenty of ways to generate hypotheses by other methods. For example, you can try to make the hypothesis space gargantuan, and encode different hypotheses in a vector of parameters, and then use different optimization or search procedures like evolutionary algorithms or gradient descent to find the most likely set of parameters. Not all of these methods are considered “Bayesian” in the sense that you are summarizing a posterior distribution over the parameters (although stochastic gradient descent might be). It seems like a full theory of intelligence might include methods for generating possible hypotheses. I think this is probably true, but I don’t know of any arguments that it would contradict Bayesian theory.

The reason assigning prior probabilities is such a huge concern is that it forces Bayesians to hold “subjective” probabilities, where in most cases, if you’re not an expert in the domain of interest, you don’t really have a good argument for why you should hold one prior over another. Frequentists often contrast this with their methods which do not require priors, and thus hold some measure of objectivity.

E.T. Jaynes never considered to this be a flaw in Bayesian probability, per se. Rather, he considered hypothesis generation, as well as assigning priors, to be outside the scope of “plausible inference” which is what he considered to be the domain of Bayesian probability. He himself argued for using the principle of maximum entropy for creating a prior distribution, and there are also more modern techniques such as Empirical Bayes.

In general, Frequentists often have the advantage that their methods are often simpler and easier to compute, while also having strong guarantees about the results, as long as certain constraints are satisfied. Bayesians have the advantage that their methods are “ideal” in the sense that you’ll get the same answer each time you run an analysis. And this is the most common form of the examples that Bayesians use when they profess the superiority of their approach. They typically show how Frequentist methods can give both “significant” and “non-significant” labels to their results depending on how you perform the analysis, whereas the Bayesian way just gives you the probability of the hypothesis, plain and simple.

I think that in general, once could say that Frequentist methods are a lot more “tool-boxy” and Bayesian methods are more “generally applicable” (if computational tractability wasn’t an issue).  That gets me to the second myth I’d like to debunk:

Being a “Strong Bayesian” means avoiding all techniques not labeled with the stamp of approval from the Bayes Council.

Does this mean that Frequentist methods, because they are tool box approaches, are wrong or somehow bad to use, as some argue that Strong Bayesians claim? Not at all. There’s no reason not to use a specific tool, if it seems like the best way to get what you want, as long as you understand exactly what the results you’re getting mean. Sometimes I just want a prediction, and I don’t care how I get it – I know that a specific algorithm being labeled “Bayesian” doesn’t confer it any magical properties. Any Bayesian may want to know the frequentist properties of their model. It’s easy to forget that different communities of researchers flying the flag of their tribe developed some methods and then labeled them according to their tribal affiliation. That’s ok. The point is, if you really want to have a Strong Bayesian view, then you also have to assign probabilities to various properties of each tool in the toolbox.

Chances are, if you’re a statistics/data science practitioner with a few years of experience applying different techniques to different problems and different data sets, and you have some general intuitions about which techniques apply better to which domains, you’re probably doing this in a Bayesian way. That means, you hold some prior beliefs about whether Bayesian Logistic Regression or Random Forests is more likely to get what you want on this particular problem, you try one, and possibly update your beliefs once you get a result, according to what your models predicted.

Being a Bayesian often requires you to work with “black boxes”, or tools that you know give you a specific result, but you don’t have a full explanation of how it arrives at the result or how it fits in to the grand scheme of things. A Bayesian fundamentalist may refuse to work with any statistical tool like that, not realizing that in their everyday lives they often use tools, objects, or devices that aren’t fully transparent to them. But you can, and in fact do, have models about how those tools can be used and the results you’d get if you used them. The way you handle these models, even if they are held in intuition, probably looks pretty Bayesian upon deeper inspection.

I would suggest that instead of using the term “Fully Bayesian” we use the phrase “Infinitely Bayesian” to refer to using a Bayesian method for literally everything, because it more accurately shows that it would be impossible to actually model every single atom of knowledge probabilistically. It also makes it easier to see that even the Strongest Bayesian you know probably isn’t advocating this.

Let me return to the “minor problems” I mentioned earlier, because they are pretty interesting.  Some epistemologists have a problem with Bayesian updating because it requires you to assume that the “evidence” you receive at any given point is completely true with probability 1. I don’t really understand why it requires this. I’m easily able to handle the case where I’m uncertain about my data. Take the situation where my friend is rolling a six-sided die, and I want to know the probability of it coming up 6. I assume all sides are equally likely, so my prior probability for 6 is 1/6. Let’s say that he rolls it where I can’t see it, and then tells me the die came up even. What do I update p(6) to?

Let’s say that I take my data as saying “the die came up even.” Then p(6 | even) = p(even | 6) * p(6) / p(even) = 1 * (1/6) / (1 / 2) = 1/3. Ok, so I should update p(6) to 1/3 now right? Well, that’s only if I take the evidence of “the die came up even” as being completely true with probability one. But what actually happened is that my friend TOLD ME the die came up even. He could have been lying, maybe he forgot what “even” meant, maybe his glasses were really smudged, or maybe aliens took over his brain at that exact moment and made him say that. So let’s say I give a 90% chance to him telling the truth, or equivalently, a 90% chance that my data is true. What do I update p(6) to now?

It’s pretty simple. I just expand p(6) over “even” as p(6) = p(6 | even) p(even)  + p(6 | odd) p(odd). Before he said anything, p(even) = p(odd) and this formula evaluated to (1/3)(1/2) + (0)(1/2) = 1/6, my prior. After he told me the die came up even, I update p(even) to 0.9, and this formula becomes (1/3)(9/10) + (0)(1/10) = 9/30. A little less than 1/3. Makes sense.

In general, I am able to model anything probabilistically in the Bayesian framework, including my data. So I’m not sure where the objection comes from. It’s true that from a modeling perspective, and a computational one, I have to stop somewhere, and just accept for the sake of pragmatism that probabilities very close to 1 should be treated as if they were 1, and not model those. Not doing that, and just going on forever, would mean being Infinitely Bayesian. But I don’t see why this counts as problem for Bayesianism. Again, I’m not trying to be omniscient. I just want a framework for working with any part of reality, not all of reality at once. The former is what I consider “One Magisterium” to mean, not the latter.

The rest of the minor issues are also related to limitations that any finite intelligence is going to have no matter what. They should all, though, get easier as access to data increases, models get better, and computational ability gets better.

Finally, I’d like to return to an issue that I think is most relevant to the ideas I’ve been discussing here. In AI risk, it is commonly argued that a sufficiently intelligent agent will be able to modify itself to become more intelligent. This premise assumes that an agent will have some theory of intelligence that allows it to understand which updates to itself are more likely to be improvements. Because of that, many who argue against “AI Alarmism” will argue against the premise that there is a unified theory of intelligence. In “Superintelligence: The Idea that Eats Smart People”, I think most of the arguments can be reduced to basically saying as much.

From what I can tell, most arguments against AI risk in general will take the form of anecdotes about how really really smart people like Albert Einstein were very bad at certain other tasks, and that this is proof that there is no theory of intelligence that can be used to create a self-improving AI. Well, more accurately, these arguments are worded as “There is no single axis on which to measure intelligence” but what they mean is the former, since even multiple axes of intelligence (such as measure of success on different tasks) would not actually imply that there isn’t one theory of reasoning. What multiple axes of measuring intelligence do imply is that within a given brain, the brain may have devoted more space to better modeling certain tasks than others, and that maybe the brain isn’t quite that elastic, and has a hard time picking up new tasks.

The other direction in which to argue against AI risk is to argue against the proposed theories of reasoning themselves, like Bayesianism. The alternative, it seems, is tool-boxism. I really want to avoid tool-boxism because it makes it difficult to be a rationalist. Even if Bayesianism turns out to be wrong, does this exclude other, possibly undiscovered theories of reasoning? I’ve never seen that touched upon by any of the AI risk deniers. As long as there is a theory of reasoning, then presumably a machine intelligence could come to understand that theory and all of its consequences, and use that to update itself.

I think the simplest summary of my post is this: A Bayesian need not be Bayesian in all things, for reasons of practicality. But a Bayesian can be Bayesian in any given thing, and this is what is meant by “One Magisterium”.

I didn’t get to cover every corollary of tool-boxing or every issue with Bayesian statistics, but this post is already really long, and for the sake of brevity I will probably end it here. Perhaps I can cover those issues more thoroughly in a future post. 

[Link] Learning from Human Preferences - from OpenAI (including Christiano, Amodei & Legg)

9 Dr_Manhattan 13 June 2017 03:52PM

View more: Next