Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Epistemic and Instrumental Tradeoffs

10 katydee 19 May 2013 07:49AM

Related: What Do We Mean By "Rationality?"

Epistemic rationality and instrumental rationality are both useful. However, some things may benefit one form of rationality yet detract from another. These tradeoffs are often not obvious, but can have serious consequences.

For instance, take the example of learning debate skills. While involved in debate in high school, I learned how to argue a position quite convincingly, muster strong supporting evidence, prepare rebuttals for counterarguments, prepare deflections for counterarguments that are difficult to rebut, and so on.

I also learned how to do so regardless of what side of a topic I was assigned to.

My debate experience has made me a more convincing and more charismatic person, improved my public speaking skills, and bolstered my ability to win arguments. Instrumentally speaking, this can be a very useful skillset. Epistemically speaking, this sort of preparation is very dangerous, and I later had to unlearn many of these thought patterns in order to become better at finding the truth.

For example, when writing research papers, the type of motivated cognition used when searching for evidence to bolster a position in a debate is often counterproductive. Similarly, when discussing what the best move for my business to make is, the ability to argue convincingly for a position regardless of whether it is right is outright dangerous, and lessons learned from debate may actually decrease the odds of making the correct decision-- if I'm wrong but convincing and my colleagues are right but unconvincing, we could very well end up going down the wrong path!

Epistemic and instrumental goals may also conflict in other ways. For instance, Kelly (2003)[1] points out that, from an epistemic rationality perspective, learning movie spoilers is desirable, since they will improve your model of the world. Nevertheless, many people consider spoilers to be instrumentally negative, since they prefer the tension of not knowing what will happen while they watch a movie.

Bostrom (2011)[2] describes many more situations where having a more accurate model of the world can be hazardous to various instrumental objectives. For instance, knowing where the best parties are held on campus can be a very useful piece of knowledge to have in many contexts, but can become a distracting temptation when you're writing your thesis. Knowing that one of your best friends has just died can be very relevant to your model of the world, but can also cause you to become dangerously depressed. Knowing that Stalin's wife didn't die from appendicitis can be useful for understanding certain motivations, but can be extraordinarily dangerous to know if the secret police come calling.

Thus, epistemic and instrumental rationality can in some cases come into conflict. Some instrumental skillsets might be better off neglected for reasons of epistemic hygeine; similarly, some epistemic ventures might yield information that it would be instrumentally better not to know. When developing rationality practices and honing one's skills, we should take care to acknowledge these tradeoffs and plan accordingly.

 

[1] Kelly, T., (2003). Epistemic Rationality as Instrumental Rationality: A Critique. Philosophy and Phenomenological Research, 66(3), pp. 612-640.

[2] Bostrom, N., (2011). Information Hazards: A Typology of Harms from Knowledge. Review of Contemporary Philosophy, 10, pp. 44-79.

New LW Meetups: Bristol, Tel Aviv

0 FrankAdamek 18 May 2013 05:15AM

The flawed Turing test: language, understanding, and partial p-zombies

8 Stuart_Armstrong 17 May 2013 02:02PM

There is a problem with the Turing test, practically and philosophically, and I would be willing to bet that the first entity to pass the test will not be conscious, or intelligent, or have whatever spark or quality the test is supposed to measure. And I hold this position while fully embracing materialism, and rejecting p-zombies or epiphenomenalism.

The problem is Campbell's law (or Goodhart's law):

The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."

This applies to more than social indicators. To illustrate, imagine that you were a school inspector, tasked with assessing the all-round education of a group of 14-year old students. You engage them on the French revolution and they respond with pertinent contrasts between the Montagnards and Girondins. Your quizzes about the properties of prime numbers are answered with impressive speed, and, when asked, they can all play quite passable pieces from "Die Zauberflöte".

You feel tempted to give them the seal of approval... but they you learn that the principal had been expecting your questions (you don't vary them much), and that, in fact, the whole school has spent the last three years doing nothing but studying 18th century France, number theory and Mozart operas - day after day after day. Now you're less impressed. You can still conclude that the students have some technical ability, but you can't assess their all-round level of education.

The Turing test functions in the same way. Imagine no-one had heard of the test, and someone created a putative AI, designing it to, say, track rats efficiently across the city. You sit this anti-rat-AI down and give it a Turing test - and, to your astonishment, it passes. You could now conclude that it was (very likely) a genuinely conscious or intelligent entity.

continue reading »

To Inspire People to Give, Be Public About Your Giving

7 peter_hurford 17 May 2013 06:58AM

Many people think it would be nicer if people were to give more money to non-profits, especially effective ones.  However, for most people, it doesn't even occur to them that they giving a large share of their salary to charity is something that people actually can do, or that people are doing on a regular basis.

Being public with one's pledge to donate not only spreads information about how easy it is to fight global poverty with a serious commitment, but that such commitments are the kind of thing that people can actually take.  By being public with these pledges, we can actually inspire people to give, where they otherwise wouldn't.

But how did people get stuck in a rut?  Why doesn't giving money come naturally?  And how would public declarations help dig people out of this rut?

 

The Bystander Effect and The Assumption of Self-Interest

First, to understand how to get people to give we have to understand why they currently do not.  There are a number of reasons, but one of the most prevalent is what's called the bystander effect.  While this effect is widely known in groups failing to respond to disasters right in front of their faces, it's magnified when the disaster is global poverty a continent or two away.  We think that because other people around us are not giving, it must also not be our responsibility, and we sure wouldn't want to be suckered into helping when no one else is doing their fair share.

Ever since Thomas Hobbes's The Leviathan, seeing human nature in terms of selfishness has been common, and persists to this day[1,2] as a strong and occasionally self-reinforcing belief[3,4].  People think of monetary incentives as being the most effective incentive for encouraging blood donations[5], even when this turns out to not be the case[6].  People greatly over-estimate the amount people will support a policy that favors them over other people[5].  As noted by Alexis de Tocqueville in 1835, "Americans enjoy explaining almost every act of their lives on the principle of self-interest"[7].

This leads us to a natural assumption that donating to charity is irrational... or, at least, other people aren't doing it, so neither should I.  However, this norm of self-interest is largely a myth, and people seem to do better than most people expect.

 

Challenging the Self-Interest Norm

This means the self-interest norm has to be challenged, and if it is challenged, we can expect people to revise their selfish-based theory of human nature and turn to more selfless acts like charitable giving.  If we're interested in getting people to donate more than what they already do, we need to open people up to the idea that charitable giving cannot only be virtuous but expected, and can be done not only at the typical rate of 1%, but at rates of 10% or much higher[8].  We also should challenge the norm that charity should be silent and not spoken about, and instead mention it openly and proudly[9].

People tend to conform, both intentionally and unintentionally, adopting the actions of others[4], and end up unwilling to adopt contrary actions unless other people are also going along with them.  If peer pressure can make high schoolers turn to drug use, alcohol drinking, cigarette smoking, or even drop out of high school[10], surely it can stop people from giving.

 

For example, take the famous Asch Conformity Experiments.  Here, people were in a group and asked to look at a line and compare its length to three other lines on another card, and state which line matches the height of the first line.  The task is enormously simple, but is complicated by being in a group of several other people, all in on the experiment, all who give the identical wrong answer.

Asch found that many people would conform to this wrong answer, even against their better judgement.  However, by adding another subject to this experiment who would give the correct answer, the tendency to conform would drop dramatically, even though the correct answer is still in the minority.  Take away the partner, even halfway through the experiment with the same subject, and conformity shoots back up.

 

However, allowing people an escape from this norm can lead them to be able to increase their charitable donations.  In one field experiment, a radio station would mention to potential donors whenever a previous donor had donated $300, and they found that this increased donations by $13 more per person over the control condition, and these donors were also more likely to renew their memberships and donate more the next year compared to those in the control condition[11].

In a separate field experiment, donors gave more to a radio station when prompted with an amount that was higher than their previous contribution[12].  Lastly, a third field experiment found that student donors were more likely to give to funds for students when told that 64% of other students had donated than when they were told that 46% of other students had donated[13].

 

Overall, people are moved by seeing what others do, and can be tilted away from self-destructive norms by seeing other people go against the flow.  An organization like Giving What We Can making a public stand for giving can accomplish just that.  Make your giving public, and it should multiply as you inspire others.

 

Motivations and Fights for Status

Reflecting on the need to push up the norm to accurately reflect the giving nature of society, it seems like the pushback to privatize giving is harmful.  And I think it is.  But why does it come about in the first place?  Robert Wiblin speculates that being public about giving calls your motivations into question.  If you're only motivated by compassion for those in need, why do you need to boast?

Well, of course, there's an interest in raising the norm.  But let's assume that giving was really just a giant fight for status... would that be so bad?  All else being equal, I prefer pure intention to that of giving just to prove to others, but competing for status via donation oneupmanship is considerably more useful than competing for status via bigger houses, bigger cars, and bigger flatscreen TVs.

Or rather, people still end up competing over their charitable contributions, but it comes in the forms of significantly less-effective (though still arguably worthwhile) charitable competition, like volunteering, building schools, or adopting African children.  If, instead, we normalized people giving checks, at least more people could be helped while the status fight goes on.

 

Conclusion

Many people want to leave the world in a better place than they found it, perhaps even going as far as wanting to do the best they can.  To these people, I hope that the idea of donation, especially to effective causes in potentially large amounts ends up appealing.  But if this cool idea is seen as "boastful", it won't catch on, and won't get the publicity (I think) it deserves.

Moreover, people won't be able to network together and share information about more cost-effective charities or the latest trends in development economics, because everyone will be keeping it to themselves, ending up being collectively self-defeating.

We seem forced by society to pretend to be self-interested, because we're asked to not talk about our acts of kindness.  But this only goes to re-enforce the deadly cycle.  The only way to push ourselves out of this cycle is to demonstrate that some people do donate and push up this norm.  And groups like GivingWhatWeCan, 80000 Hours, and BolderGiving are working on doing just that.

Personally, I'd have to agree that this works -- I'm inspired by these stories, and I don't think I would ever be donating 10%+ without a group that makes it seem like a completely normal and awesome thing to do.

So is talking about donations too boastful?  I think, for the sake of those the donations help, we can afford a little boasting in this one area.

 

References and Notes

(Note: Most of these links open to PDFs.)

[1]: Barry Schwartz. 1986. The Battle for Human Nature: Science, Morality and Modern Life. Canada: Penguin Books.

[2]: Alfie Kohn. 1990.
The Brighter Side of Human Nature. New York: Basic Books.

[3]: Dale T. Miller. 1999. "The Norm of Self-Interest". American Psychologist 54 (12): 1053-1060.

[4]: John M. Darley and Russell H. Fazio. "Expectancy Confirmation Processes Arising in the Social Interaction Sequence". 1980. American Psychologist 35 (10): 867-881.

[5]: Dale T. Miller and Rebecca K. Ratner. 1998. "The Disparity Between the Actual and Assumed Power of Self-Interest". Journal of Personality and Social Psychology 74 (1): 53-62.

[6]: Nicola Lacetera, Mario Macis, and Robert Slonim. 2011. "Rewarding Altruism? A Natural Field Experiment". The National Bureau of Economic Research Working Paper #17636.

[7]: Alexis de Tocqueville in J.P. Mayer ed., G. Lawrence, trans. 1969. 
Democracy in America. Garden City, N.Y.: Anchor, p546.

[8]: The Giving What We Can pledge requires 10% and this is already shockingly high for most, but people on 80000 Hours's member list or among Bolder Giving's stories donate up to 50% of their income or more!

[9]: Of course, I don't think we should mention it *all* the time -- we should recognize when is the time and place, and not be unreasonable.  On the same time, we shouldn't be completely silent.  Places like Facebook, personal blogs, and when the topic comes up for conversation all seem like fair game.

[10]: Alejandro Gaviria and Steven Raphael. 2001. "School-Based Peer Effects and Juvenile Behavior". The Review of Economics and Statistics 83 (2): 257-268.

[11]: Other conditions were $180, $75, or no prompt about previous donors at all.  Jen Shang and Rachel Croson. Forthcoming. “Field Experiments in Charitable Contribution: The Impact of Social Influence on the Voluntary Provision of Public Goods”. The Economic Journal.

[12]: Rachel Croson and Jen Shang. 2008. "The Impact of Downward Social Information on Contribution Decisions"Experimental Economics 11: 221-233.

[13]: Bruno S. Frey and Stephan Meier. 2004. "Social Comparisons and Pro-social Behavior: Testing 'Conditional Cooperation' in a Field Experiment". The American Economic Review 94 (5): 1717-1722.

-

Also cross-posted on my blog.

Post ridiculous munchkin ideas!

39 D_Malik 15 May 2013 10:27PM

Thus spake Eliezer:

A Munchkin is the sort of person who, faced with a role-playing game, reads through the rulebooks over and over until he finds a way to combine three innocuous-seeming magical items into a cycle of infinite wish spells.  Or who, in real life, composes a surprisingly effective diet out of drinking a quarter-cup of extra-light olive oil at least one hour before and after tasting anything else.  Or combines liquid nitrogen and antifreeze and life-insurance policies into a ridiculously cheap method of defeating the invincible specter of unavoidable Death.  Or figures out how to build the real-life version of the cycle of infinite wish spells.

It seems that many here might have outlandish ideas for ways of improving our lives. For instance, a recent post advocated installing really bright lights as a way to boost alertness and productivity. We should not adopt such hacks into our dogma until we're pretty sure they work; however, one way of knowing whether a crazy idea works is to try implementing it, and you may have more ideas than you're planning to implement.

So: please post all such lifehack ideas! Even if you haven't tried them, even if they seem unlikely to work. Post them separately, unless some other way would be more appropriate. If you've tried some idea and it hasn't worked, it would be useful to post that too.

How to Build a Community

9 peter_hurford 15 May 2013 05:43AM

I've noticed that quite a few people are interested in fostering communities -- both creating communities and improving them to make them work together.  But how do we go about actually doing this?  What's there to community that we can foster and build upon?  What makes a community thrive, and how do we take advantage of this to make and/or improve communities?

To answer these questions, I turned to two books:

The first is The Penguin and The Leviathan: How Cooperation Triumphs Over Self-Interest by Yochai Benkler.  Benkler, in writing about cooperative systems (Penguins, named after the Linux Penguin) and hierarchical systems (Leviathans, named after Thomas Hobbes's The Leviathan), studies the psychology, economics, and political science of cooperation and helps explain what makes communities stick.

The second is Liars and Outliers: Enabling the Trust that Society Needs to Thrive by Bruce Schneier.  Schneier studies trust and cooperation from a dizzying variety of sciences (psychology, biology, economics, anthropology, computer science, and political science).  Schneier's ultimate game is figuring out what is preventing society from falling apart, and that can be applied to building communities.

Let's see what they got.

 

Communities Need Cooperation

Schneier and Benkler both paint a view of human nature that is different than what is commonly thought, but what has emerged from the sciences: People are both self-interested and other-interested, different people will have different balances of each, and within each person these two goals can often conflict.  Additionally, the "other-interested" aspect can be multiple and occasionally conflicting allegiances, such as to one's family, to one's neighborhood, to one's country, to one's venture philanthropy club, etc.

What's unique to all communities is that they involve people who have set aside some of their immediate self-interest to work together.  For instance, when we work together in a group, I definitely don't beat you over the head and steal your lunch money, and I don't usually attempt to free ride and get you to do the group work for me, but we mutually work to solve communal problems and share in the benefits of community.

Public Goods and Free Riders

An example of how psychology has sought to simplify and simulate a community is through what's called "The Public Goods Game".  In this game, a group of about ten participants are each sat down, and given $10 each to start with.  The game is then played for several rounds, and in each round all participants get to put a certain secret amount of their money into a collective pot.  The experimenters then look at the pot, double the amount of money inside it, and redistribute the result evenly to all the players.  For added bonus, the experimenters inform all participants that they get to walk away with their winnings after the game is over.

If everyone went perfectly with the community, each player would see their money double each round.  But the wrinkle is that if people don't contribute at all to the pot, then they stand to gain even more money from the results of everyone else's contributions.  This is called the free rider problem: there is a tension between wanting to contribute to the pot for the good of yourself and the good of the group as a whole and refraining from contributing so that you benefit even more.

The Free Rider Problem and The Collective Action Problem

But the tension can result in further disaster, for imagine everyone decides to be a free rider and defect from the group -- now, no money goes in the pot at all, and everyone ends with the $10 they start with.  This gets worse when we imagine some other real-life scenarios -- for instance, that of fishermen in a lake.

The fishermen can either choose to fish normally or overfish.  If all the fishermen overfish, they stand to deplete the lake and all fishermen lose their jobs.  However, if just a few fishermen overfish, they get the benefit of added fish to sell, and the lake can handle the slight increase in load.  So this tension is to be the fisherman that wins most by personally overfishing, while not collectively depleting the entire lake.  Such problems are called collective action problems -- people do well individually by defecting but do worse collectively if everyone defects.  The result of a collective action problem ending in disaster is called the tragedy of the commons.

The Community Solution

So what's the solution to these problems?  Benkler proposes two models for dealing with them -- employing the Leviathan and placing lots of regulations on overfishing and enforcing them with strict punishments, or employing the Penguin and creating a community that deals with these problems collectively and in a self-policing way.

It turns out that certain problems are best dealt with differing combinations of Leviathan and Penguin models, but most problems need lots of community just because it can be difficult to figure out who is going against the community, and communities have more freedom for their participants.  At the same time, if there are too many would-be defectors a community can never get off the ground.

Communities need cooperation to work.  So how can we get this cooperation to fly?

 

The Four Pressures of Cooperation

Bruce Schneier notes that normally we don't think through these free rider problems and try to scheme our way through them -- we just cooperate, instinctively.  We don't assume people will rip us off, and we usually don't rip other people off -- that's just how we are.  But why?  Schneier suggests that cooperation can be fostered and maintained through four different pressures, though differing kinds and amounts of pressure apply to different situations, and getting the balance of pressures right is a key part of his book:

1.) Moral Pressures: Many, but not all of us, have various moral feelings that lead us to want to cooperate.  It could be as easy as feeling incredibly guilty when we defect against our friends, or as complex as subscribing to an abstract principle of justice.  For most of us, it's a general feeling that cooperating is the "right thing to do" and defecting for our own personal self-interest is "wrong", and we just don't want to do it.  Schneier and Benkler both find that moral pressures compel cooperation a surprising amount of the time.

2.) Reputational Pressures: Another part about living in a community for a long time is that you have a reputation to live by.  Defect against the community and you may win a few times, but then people start to notice and start working to stop you.  They might refuse you friendship or other things you want, or even kick you out of the community altogether!  Benkler finds that many communities can thrive on reputation alone, like eBay, Amazon, or Reddit.

3.) Institutional Pressures: Morals and reputation aren't the end of it though; many communities make specific, codified norms and enforce them with specific, codified punishments.  These pressures are laws, and the fear of breaking the law, being caught, and getting the punishment can often further spur cooperation.  Best yet, the community can often get together and agree to these norms, realizing it is in their individual benefit to force themselves and the rest of the community to play along, as to avoid tragedies of the commons.

4.) Security Pressures: Lastly, there are always going to be a few people who put morals, reputation, and laws aside and try to defect anyway.  For these, we hope to stop them in their tracks or make their jobs more difficult, by using complex security systems.  It can be as simple as a security camera or anti-theft radio, or as complex as Fort Knox.  Security is a double plan: it first attempts to raise the costs of defection; by making it physically harder to defect, one is less tempted to do so.  It then attempts to better catch and apprehend those who still try.

 

Your Reason for Joining; Your Reason for Staying

Remember these pressures don't all work for the same problems -- it may be proper to use security and institutional pressures to stop someone from overfishing, but not from intentionally cutting the cake so they get to eat the bigger slice.  Moral and reputational pressures seem to be more encompassing, but they are also more easily defeated -- people with less of a moral compass can often wander from community to community, wrecking small amounts of havoc and never getting caught or punished.

Benkler suggests another way to get people to buy into a community and not defect against it -- make it clear that being part of the community is something they really want.  Whether your joining a community or forced into one (family, country, etc.), the community will be more likely to thrive.

Four Ways to Bond

But why might one want to join or stay in a community?  For many, the answer is the intangibles -- they feel a sense of belonging, friendship, and group cohesion that creates an empathetic attachment and makes people want to play by the rules of the group.  For others, the answer is the tangibles -- the group may have a stated mission statement that is important to the person, or belonging in the group might confer a specific benefit.  People might even belong for a mix of tangibles and intangibles, plus a natural tendency to want to join groups.

But how do we foster these bonds?  Benkler has his own set of four things, suggesting that group identity can be fostered through a combination of four means:

1.) Fairness: The community needs to be fair -- people need to all contribute more or less equally, or at least have genuine intentions to put in equal effort, and the benefits of the group need to be spread among all participants more or less evenly, or in a fair proportion to how much the participant puts in.

2.) Autonomy: The community needs to not demand too much, and make sure to compensate quickly and generously for special sacrifices.  There are inherent costs to joining and staying with a group, and costs for cooperating with the group -- one doesn't just give up the self-interested benefits of defection, but rather must pay additional costs to maintain their group status.  Being aware of and addressing these costs are important.  In short, the group must respect their members as individuals.

3.) Democracy: The community also needs to accept (with fairness and autonomy) the input of all the members.  Group norms should be developed by a vote, with weight given on building consensus as much as possible, and with understanding the reasons why people might not like the consensus.  Not only does having input make it more likely people's preferences will be taken into account, lowering the costs of cooperation, but having input makes people feel more group cohesion and belonging.

4.) Communication: During times when formal votes aren't taken, the community also needs to be consistently (but not constantly) talking about how the group is doing, and checking in with members who might be feeling left out.  Just like democracy, group cohesion is built through communication, and communication lowers the costs of cooperation.  It's best when resolving disputes is not dictatorial, like in a court of law, but rather cooperative, like in an arbitration.

 

Looking Back to the Public Goods Game

To demonstrate these four points, Benkler draws on many real-world examples, such as policies of various companies, and interactions on the internet.  He also draws on returning back to our simple-community-in-the-lab, the Public Goods Game, for additional confirmation, and its worth seeing how these things play out.

In the original Public Goods game, contributions to the pot were made anonymously and no-one was allowed to talk or communicate.  Typically, a fair amount of people would cooperate in the beginning (generally, people contribute about 70% of their share), but starts to drop as people see that others aren't contributing.  They start to feel like suckers, and the fairness starts to kick in.

A Different Game

However, variants of the Public Goods game offer ways out.  When participants were allowed to talk to each other, contributions rose (communication).  Likewise, when participants were allowed to use some of their money to punish those who didn't contribute (say, pay $3 to prevent someone from getting their share this round if they didn't cooperate last round), people would do so.  

Even the simple act of making the contributions public increased cooperation, drawing on reputation.  Sometimes small fines were imposed on those who didn't cooperate (institutional pressures) which brought up cooperation, and these fines worked especially well when the group got to vote on how high they would be (democracy).

Lastly, helping frame the game would help -- those who were told they were taking place in a "Community Game" were far more likely to contribute to the pot and keep contributing than those who were told they were taking place in a "Wall Street Game".  By reminding people they are in a community, people thought more about their community norms, and felt more group cohesion, and were more likely to trust others.

 

Conclusions

Ultimately, creating communities is all about fostering cooperation, and you foster cooperation by ensuring that there is mutual trust and some sort of way to prevent defectors from taking advantage of the system.  People often naturally don't want to defect, but will do so if they think others will take advantage of them first.

Social Pressures

But how do we foster this trust?  The first step is to make use of our social pressures when and to the amount that's appropriate -- relying on empathetic and moral norms, reputation, institutionalized laws, and security systems -- and being sure to get the balance right.  For small communities, this probably just needs to be a set of agreed norms, and ensuring that the norms are properly and responsibly enforced.

The Benefits of Joining

The second part is while implementing the first step, we should keep in mind why people are joining or staying in the first place, and make sure to provide a community where the benefits of joining -- both the tangibles and intangibles -- are present and apparent.  We should acknowledge the costs of cooperating, and make sure the benefits are there to foster group loyalty and belonging.

An Effective Community

While implementing, it's important to keep in mind that communities should also be fair, respect the autonomy and individuality of the members, give members input through democracy, and foster lots of communication about how things are going.  We should also keep a keen eye to how things are framed, while not going overboard on it or lying.

The End Reward

But when we accomplish communities, the rewards are pretty great -- not only do we avoid free riders and the tragedy of the commons, but we ourselves get to take advantage of communities that are more productive than the individuals alone, and secure the feelings of belonging to a group we enjoy.

-

Also cross-posted on my blog.

 

The Power of Pomodoros

42 elharo 14 May 2013 10:36AM

Until recently, I hadn't paid much attention to Pomodoro, though I've heard of it for a few years now. "Uncle Bob" Martin seemed to like it, and he's usually worth paying attention to in such matters. However, it mostly seemed to me like a way of organizing a variety of tasks and avoiding procrastination, and I've never had much trouble with that.

However after the January CFAR workshop suggested it in passing, I decided to give it a try; and I realized I had it all wrong. Pomodoros aren't (for me) a means of avoiding procrastination or dividing time among projects. They're a way of blasting through Ugh fields.

The Pomodoro technique is really simple compared to more involved systems like Getting Things Done (GTD). Here it is:

  1. Set a timer for 25 minutes
  2. Work on one thing for that 25 minutes, nothing else. No email, no phone calls, no snack breaks, no Twitter, no IM, etc.
  3. Take a five minute break
  4. Pick a new project, or the same project, if you prefer.
  5. Repeat

That's pretty much it. You can buy a book or a special timer for this; but there's really nothing else to it. It takes longer to explain the name than the technique. (When Francesco Cirillo invented this technique in the 1980s, he was using an Italian kitchen timer shaped like a tomato. Pomodoro is Italian for tomato.)

I got interested in Pomodoro when I realized I could use it to clean my office/desk/apartment. David Allen's GTD system appealed to me, but I could never maintain it, and the 2+ days it needed to get all the way to a clean desk was always a big hurdle to vault. However, spending 25 minutes at a time, followed by a break and another project seemed a lot more manageable.

I tried it, and it worked. My desk stack quickly shrunk, not to empty, but at least to a place where an accidental elbow swing no longer launched avalanches of paper onto the floor as I typed.

So I decided to try Pomodoro on my upcoming book. The publisher was using a new authoring system and template that I was unfamiliar with. There were a dozen little details to figure out about the new system--how to check out files in git, how to create a section break, whether to use hard or soft wrapping, etc.--and I just worked through them one by one. 25 minutes later I'd knocked them all out, and was familiar enough with the new system to begin writing in earnest. I didn't know everything about the software, but I knew enough that it was no longer averting. Next I used 25 minutes on a chapter that was challenging me, and Pomodoro got me to the point where I was in the flow.

That's when I realized that Pomodoro is not a system for organizing time or avoiding procrastination (at least not for me). What it is, is an incredibly effective way to break through tasks that look too hard: code you're not familiar with, an office that's too cluttered, a chapter you don't know how to begin.

The key is that a Pomodoro forces you to focus on the unfamiliar, difficult, aversive task for 25 minutes. 25 minutes of focused attention without distractions from other, easier tasks is enough to figure out many complex situations or at least get far enough along that the next step is obvious.

Here's another example. I had a task to design a GWT widget and plug it into an existing application, and I have never done any work with GWT. Every time I looked at the frontend application code, it seemed like a big mess of confused, convoluted, dependency injected, late bound, spooky-action-at-a-distance spaghetti. Now doubtless there wasn't anything fundamentally more difficult about this code than the server side code I have been writing; and if my career had taken just a slightly different path over the last six years, frontend GWT code might be my bread and butter. But my career didn't take that path, and this code was a big Ugh field for me. So I set the Pomodoro timer on my smartphone and started working. Did I finish? No, but I got started, made progress, and proved to myself that GWT wasn't all that challenging after all. The widget is still difficult enough and GWT complex enough that I may need several more Pomodoros to finish the job, but I did get way further and learn more in 25 minutes of intense focus than I would have done in a day or even a week without it.

I don't use the Pomodoro technique exclusively. Once I get going on a project or a chapter, I don't need the help; and five minute breaks once I'm in the flow just distract me. So some days I just do 1 or 2 or 0 Pomodoros, whatever it takes to get me rolling again and past the blocker.

I also don't know if this works for genuinely difficult problems. For instance, I don't know if it will help with a difficult mathematical proof I've been struggling with for months (though I intend to find out). But for subjects that I know I can do, but can't quite figure out how to do, or where to start, the power of focusing 25 minutes of real attention on just that one problem is astonishing.

Pascal's Muggle: Infinitesimal Priors and Strong Evidence

38 Eliezer_Yudkowsky 08 May 2013 12:43AM

Followup to:  Pascal's Mugging: Tiny Probabilities of Vast Utilities, The Pascal's Wager Fallacy Fallacy, Being Half-Rational About Pascal's Wager Is Even Worse

Short form:  Pascal's Muggle

tl;dr:  If you assign superexponentially infinitesimal probability to claims of large impacts, then apparently you should ignore the possibility of a large impact even after seeing huge amounts of evidence.  If a poorly-dressed street person offers to save 10(10^100) lives (googolplex lives) for $5 using their Matrix Lord powers, and you claim to assign this scenario less than 10-(10^100) probability, then apparently you should continue to believe absolutely that their offer is bogus even after they snap their fingers and cause a giant silhouette of themselves to appear in the sky.  For the same reason, any evidence you encounter showing that the human species could create a sufficiently large number of descendants - no matter how normal the corresponding laws of physics appear to be, or how well-designed the experiments which told you about them - must be rejected out of hand.  There is a possible reply to this objection using Robin Hanson's anthropic adjustment against the probability of large impacts, and in this case you will treat a Pascal's Mugger as having decision-theoretic importance exactly proportional to the Bayesian strength of evidence they present you, without quantitative dependence on the number of lives they claim to save.  This however corresponds to an odd mental state which some, such as myself, would find unsatisfactory.  In the end, however, I cannot see any better candidate for a prior than having a leverage penalty plus a complexity penalty on the prior probability of scenarios.

In late 2007 I coined the term "Pascal's Mugging" to describe a problem which seemed to me to arise when combining conventional decision theory and conventional epistemology in the obvious way.  On conventional epistemology, the prior probability of hypotheses diminishes exponentially with their complexity; if it would take 20 bits to specify a hypothesis, then its prior probability receives a 2-20 penalty factor and it will require evidence with a likelihood ratio of 1,048,576:1 - evidence which we are 1048576 times more likely to see if the theory is true, than if it is false - to make us assign it around 50-50 credibility.  (This isn't as hard as it sounds.  Flip a coin 20 times and note down the exact sequence of heads and tails.  You now believe in a state of affairs you would have assigned a million-to-one probability beforehand - namely, that the coin would produce the exact sequence HTHHHHTHTTH... or whatever - after experiencing sensory data which are more than a million times more probable if that fact is true than if it is false.)  The problem is that although this kind of prior probability penalty may seem very strict at first, it's easy to construct physical scenarios that grow in size vastly faster than they grow in complexity.

I originally illustrated this using Pascal's Mugger:  A poorly dressed street person says "I'm actually a Matrix Lord running this world as a computer simulation, along with many others - the universe above this one has laws of physics which allow me easy access to vast amounts of computing power.  Just for fun, I'll make you an offer - you give me five dollars, and I'll use my Matrix Lord powers to save 3↑↑↑↑3 people inside my simulations from dying and let them live long and happy lives" where ↑ is Knuth's up-arrow notation.  This was originally posted in 2007, when I was a bit more naive about what kind of mathematical notation you can throw into a random blog post without creating a stumbling block.  (E.g.:  On several occasions now, I've seen someone on the Internet approximate the number of dust specks from this scenario as being a "billion", since any incomprehensibly large number equals a billion.)  Let's try an easier (and way smaller) number instead, and suppose that Pascal's Mugger offers to save a googolplex lives, where a googol is 10100 (a 1 followed by a hundred zeroes) and a googolplex is 10 to the googol power, so 1010100 or 1010,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000 lives saved if you pay Pascal's Mugger five dollars, if the offer is honest.

continue reading »

Maximizing Your Donations via a Job

85 Alexei 05 May 2013 11:19PM

In November of 2012 I set a goal for myself: find the most x-risk reducing role I can fill. At first I thought it would be by working directly with MIRI, but after a while it became clear that I could contribute more by simply donating. So my goal became: find the highest paying job, so I can donate lots of money to CFAR and MIRI.

A little bit of background on me. Started programming in 2000. Graduated in 2009 with Bachelor's in computer science. Worked for about a year and a half at a game company. Then did my own game startup for about a year. Then moved to the bay area and joined a game startup here, which was acquired 10 months later. Worked a bit at the new company and then left. So, just under four years of professional programming experience, but primarily in the game industry. Almost no leadership / managerial experience, aside from the startup I did where I hired freelancers.

Below is my experience of finding a software engineering job in the Silicon Valley. If you are not an engineer or not in the Silicon Valley, I think you'll still find a lot of useful information here.

 

Pre-game

Before sending out my resume, I spent about a month preparing. I read Intro to Algorithms, which was very good overall, but not a huge help in preparing for interviews.[1] I read Cracking the Coding Interview, which was extremely helpful. (If you read only one book to prepare, make it this one.) The book has a lot of questions that are similar to the ones you'll actually see during interviews. I also did TopCoder problems, which were pretty helpful as well.[2] Looking back, I wish I spent more time finding actual interview questions online and doing more of those (that's why CCI book was so helpful).

After several weeks of preparation, I compiled a long list of companies I was going to apply to. I checked on GlassDoor to see what kind of salary I could expect at each one. I then rated all the companies. Companies with low salaries and poor personal fit received the lowest rating.

I started by applying to companies with the lowest ratings. This way I could use them as practice for the companies I thought would actually make a competitive offer. This was the right move and worked very well. (Another friend of mine did the same approach with good results as well.) Remember, you are not just doing those interviews to practice the coding problems, you are practicing pitching yourself as well.

 

Interviewing with a company

Standard procedure for applying to a tech company:

1. Send them your resume.

  • Proofread your resume. Let your friends proofread it.
  • Make sure there are only relevant things on it. When I applied to tech companies, I removed a lot of game-specific things from my resume. When I applied to companies that did 3D graphics, I made sure I had all my 3D graphics experience listed. I ended up with two version of my resume.
  • Have your resume in DOC, PDF, and TXT formats. This way you'll always have the right one when you upload / paste it.
  • For a few companies, I had a friend or friend of a friend who referred me. This REALLY HELPS in two ways: 1) your resume will be processed a lot faster, 2) if your friend is a great engineer/employee, you'll be taken a lot more seriously, and the company will fight for you a lot harder.

2. You'll get an email from the recruiter and setup a time to speak, where you'll talk about yourself, what you've done, why you are interested in their company, and so on. You can and should ask them questions as well.

  • When you start getting multiple calls each day, make sure you know who is calling. There is nothing worse than talking about the challenges of streaming music to a car sharing startup. (True story.)
  • Read about the company on Wikipedia before the call. Know the basic stuff. Look at their website and read the About page.
  • Find the thing that makes the company special and successful. Find the thing that you actually think is cool about the company. Those are your answers for why you want to work there.
  • Ask non-technical questions: How is the company structured? How many teams are there? How many employees? Engineers? Think of other intelligent questions to ask.
  • In my experience, it's not very beneficial to tell them you are interviewing with a dozen other companies. When they ask who else you are interviewing with, just name a few companies, especially the competitors / similar companies.
  • Be SUPER NICE to your recruiter. They are your main point of contact with the company. They'll be the one fighting to get you the best offer.

3. You'll have a technical phone interview with a software engineer where you'll solve a problem or two on collabedit or some similar website. At the end, you'll get a few minutes to ask them questions too.

  • All the usual interviewing tips apply here. E.g. talk out loud, your interviewer doesn't know what you are thinking.
  • Most companies don't care what language you use, as long as it's mainstream. (I used C# for almost all my coding questions.)
  • DO NOT start answering the question by writing code. If the questions seems vague, ask about the context. Who'll be using this solution? Definitely ask about the kind of data you are working with. If it's integers, are they random? Over some small range or over all possible integers?
  • List out metrics for various approaches: brute-force solution, optimized for speed solution, optimized for memory solution. Here is a question I saw a few times: Write a data structure which can accept and store integers, and can check if there exist two integers that sum up to a given number. There are multiple solutions, and the best one depends on the ratio of addInteger to checkForSum calls.
  • The previous steps should only take you a minute or two. Once you've decided what the best approach is, then you can write the solution. When you are done, check for errors, then run through several examples. Do a simple example and a slightly complicated example. When you find a bug, don't be hasty in fixing it. Understand why it happened and make sure you won't introduce new bugs by fixing it.
  • If everything works, make sure you handle errors correctly. Can you handle invalid input? Input that violates your assumptions? (As a reminder, I leave “\\Check for errors” comments in appropriate spots as I code the solution.)
  • When you are done, ask the interviewer questions. Ask them to tell you about what they do, if they haven't already. What have they been working on recently? What technologies/languages do they use at the company? Do they use Scrum/Agile? Pair-programming? Come up with other intelligent questions to ask.

 

4. You'll be invited for an on-site interview which will be 3-6 hours long, at least half of which will be coding on a white-board. (Although, a friend told me he brought his laptop with him, and most people were fine with him coding on it.)

  • All the previous tips apply.
  • Be on time. Take bathroom breaks when you need them. I found that drinking water during the interview keeps me refreshed. Remember your posture, body-language, and eye-contact skills.
  • Learn how to talk out loud as you are writing out your solution. If you are stuck, explain what you are thinking, and what your intuition is telling you.
  • Learn how to read your interviewers. If you say, "Here we should check for the null case or for empty array," and they go "Yeah, yeah, okay," they are not the type of interviewer that really cares about error conditions, so you can be somewhat more lax there. By the time I was finishing my on-site interviews, I could tell if my solution was right just by the interviewer's body language.
  • When you are done, ask them questions. What are they working on? What's the thing they like most about the company? What's their least favorite thing about the company? (Another way to phrase that: What's one thing you would change if you could change anything about the company?) Do they have to work overtime? How are the people here? Can you switch between projects? Are there company wide events? In all my interviews I've never met an interviewer that didn't try to sell their company really hard. People will always tell you their company is the best place to work.
  • If the person is a manager or a director, ask them higher level questions. What kind of culture are they trying to create? What are the current big challenges? Where do they want the company to be in the next 5 years? How does one advance in the company? (Usually there is a managerial and a technical track.) How often are reviews done? How are they structured?

 

5. You'll get a call from the recruiter congratulating you on an offer. They'll go over the offer details with you.

  • Before they make you an offer, they'll check if you are actually seriously considering their company. If you told a startup you are also interviewing with Google, they might suspect that you are not seriously considering them. Unless you dissuade those fears, they might actually not even make you an offer. (Happened to me with Rdio.)
  • If you didn't get an offer, try to get as much info as you can. What happened? What can you improve on? Below are the reasons why I didn't get an offer after an on-site interview:
    • Not doing well on a technical question. (Happened twice; one time because of a very obnoxious interviewer.)
    • Not interviewing for quite the right position (that on-site interview ended early).
    • Not having the necessary experience (a lot more important to startups than bigger companies).
    • Not being passionate enough about the company.
  • If this is not a good timing for the offer, e.g. it's one of your first interviews, then tell them so. They will probably wait to give you the offer details until you are ready to consider it.
  • The recruiter will likely ask what's important to you in an offer. How are you going to make your decision? What I've said is that compensation will be an important factor in my decision, but that the team/project/etc. are important considerations as well.

 

6. You have a few days (usually around 5 business days) until the offer expires to decide if you want to accept it.

  • Sometimes the offer will expire before you've received offers from other companies. This is why it's important to interview in rough order of ranking, so that you can just let those offers go, knowing you'll have much better ones soon. If you want to hold on to the offer, just ask your recruiter for an extension. It'll be much easier to get an extension at big companies, especially if you are interviewing for a generic position.
  • If you decline the offer, let them know.

 

Always be very nice, friendly, and polite. Walk the fine line between telling the truth and saying the right thing. Ideally, make sure those are the same. Even if you are interviewing with a company you have no intention of working at, make sure to find something you really like about them, something that makes them stand out to you. Always have a good answer to: "Why do you want to work here?"

Before each on-site interview make sure you research the company thoroughly. Use their product. Think of ways to improve it. It's very helpful if you can meet with someone that works there and talk to them. See if they can give you any tips on the interview process. Some companies (e.g. AirBnB) want people that are extremely passionate about their product. Some companies focus more than usual on architectural questions. Many companies expect the engineers to have some familiarity with UI/UX and the ability to think about a feature from all angles.

 

Managing your time

I sent my resume to 78 companies, had at least a phone conversation with a recruiter with 27 of them, had an on-site interview with 16 companies, and received 12 offers. Out of those, I've only seriously considered 3. (Companies with lower ratings had an atrocious response rate.)

My time-line ended up looking something like this:

  • Week 1: Started applying to low-rated companies. About 2 phone interviews.
  • Week 2: About 7 phone interviews. One on-site interview. Sending out more resumes.
  • Week 3: About 3 phone interviews.
  • Week 4: About 15 phone interviews. A few meetings with friends of friends, who ended up referring me. 1 on-site interview. Sent my resume to all the high-rated companies. (During this week interviewing became a full-time job.)
  • Week 5: About 10 phone interviews. 4 on-site interviews.
  • Week 6: 8 phone interviews. 4 on-site interviews.
  • Week 7: 4 phone calls. 5 on-site interviews.
  • Week 8: 12 phone calls. 2 on-site interviews.
  • Week 9: About 8 calls a day for a few days, while I negotiated with my top companies.
  • (These are strictly lower bounds for phone calls. On-site data is pretty accurate.)

Some companies move fast, some companies move slow. Google took 2 weeks from the on-site interview to the offer call. This is very common for them, but most other companies move faster. With Amazon, I actually interviewed with two different branches. With one branch things were going well, until they dropped the ball and never got back to me, even after I pestered them. This is unusual; although, Twitter did something similar, but then ended up responding with an on-site invitation. With the other Amazon branch, when I got home from the on-site interview, I already had an email saying they were going to make an offer. This is extremely fast. (I had a very good reference for that position.) Most companies take about a week between on-site and offer. The whole process, from first call to offer, takes about three weeks.

If your recruiter doesn't respond to you during 4 days or longer, shoot them an email. They might have forgotten to respond, or thought they did, or may be things are moving slowly, or may be they decided not to pursue. You want to be clear on where you stand with all the companies you are applying to.

The timing is pretty important here. You want your top-rated companies to give you an offer within a span of a week. This way you'll be able to leverage all those offers against each other.

If your current job position is already almost optimal for your goals, then it's possible you can do a few interviews, get a few offers and pick the best one, which will give you some marginal improvement. Or use those offers to leverage a raise at your existing company. But if you are pretty sure your current job has not been optimized for your goal, then I'd say, contrary to popular wisdom, just leave and spend a full month interviewing. (Or, even better, if you can, take a long "vacation".) You just can't do this kind of intense interviewing while holding another job. The one exception to this rule I can think of is if one of your highest-rated companies is a competitor with your current employer. Then you can leverage that!

Value of information is extremely high during this process. Talk to all the companies you can, talk to all the people you can. Once you have the final list of companies you are considering, reduce your uncertainty on everything. Validate all your assumptions. (Example: I was sure Google matched donations up to $12k, but turns out it's only up to $6k.)

 

How to evaluate your offer

There are 4 basic components in an offer: sign-on bonus, base salary, equity, and bonus.

Sign-on bonus. Most companies will be okay offering something like $12k sign-on bonus. Some will offer more. Most startups probably won't offer any.

Base salary. This is pretty consistent across most companies. Based on your experience, you'll be given a title (e.g. Senior Software Engineer or SE 2), and that title will determine the range of the salary you can expect. If you are good, you can demand a salary at the top of that range, but it's extremely hard to go higher.

Equity. This is the most interesting part. A good amount of value will come from this portion. With a startup, it'll be most of it. Here are two things to pay attention to:

  • Is the company public or private? If it's public, you are most likely going to be given RSUs (restricted stock units), which will basically convert to normal company shares when they vest. For private companies, see the section below.
  • What's the vesting schedule? For almost all companies you'll get 25% of your shares right after your first year. (This is called a 'cliff'.) After that you'll be given the appropriate fraction either monthly (e.g. at Google) or quarterly (e.g. at Facebook). Amazon is an example of a company where the vesting schedule is somewhat different: 5% after year 1, 15% after year 2, and then 20% each semester for the next two years.

Bonus. This is the bonus system the company has setup. You can't negotiate it, but it's important to take it into account.

  • There will usually be a cash bonus that's based on your salary. It'll have a target percent (e.g. 15%). If you can find out how many people hit their target, that will be very helpful. However, most companies don't share or simply don't have that information.
  • Some companies also have equity bonuses. Try to get as much info on those as you can. Don't assume that you'll get the maximum bonus even if you work hard. If you have friends working at that company, ask them what kind of bonuses they've been getting.
  • Lots of startups don't have bonus systems in place.

Other factors.

  • Donation matching: Google matches up to $6k (you donate $6k to any charity, they'll donate another $6k). Craigslist matches 3:1 up to 10% of your salary. Most companies don't have anything like that, and you can't negotiate it.
  • Paid Time Off: Google offers 2 weeks, all other companies I was considering offer 3 weeks, and some even have unlimited PTO. This is not negotiable in most companies.
  • Commute: how far will you have to travel to work? Are you okay moving closer to work? (Google and Facebook have shuttles that can pick you up almost anywhere, so you could work while you commute.)
  • People/culture/community/team/project are all important factors as well, depending on what you want. If you are going to spend the next several years working on something, you should be building up skills that will still be valuable in the future.

 

Thinking about private companies

If the company is private, you might be given RSUs or you might be given stock options. With stock options, you'll have to pay the strike price to exercise your options. So the total value your options have is: (price of a share - strike price) * number of shares.

You can't do anything with your shares until the company gets acquired or goes public. Some companies have liquidation events, but those are pretty rare. Most companies don't have them, and the ones that do only extend the opportunity to people that have been with the company for a while. There are also second-hand markets, but I don't know much about those.

If you are completely risk-intolerant, then just go with a public company, and don't consider private companies. (This is actually not exactly true. Just because a company is public, doesn't mean its risk-free, and just because a company is private doesn't mean there is a lot of risk. There are other important factors like the size of the company, their market diversity, and how long they've been around.) If you are okay with some risk, then you want a company that's close to an IPO or is likely to get acquired soon. If you want to have a chance to make more than a few million dollars, either start your own company or join a very early stage startup (my top pick would be Ripple). Before doing so, check out the stats on startups to make sure you understand how likely any given startup is to fail and make sure you understand the concepts of inside/outside view.

 

Taxes

It's crucial to understand all the tax implications of your salary, equity, and donations. I'm not going to go into all the details, there are a lot of resources out there for this, but you should definitely read them until it's crystal clear how you will be taxed. I'll highlight a few points:

  • Understand the tax rate schedule and notice the new 39.6% tax bracket. If your income is $100k, that doesn't mean you get taxed 28% on all of it. 28% applies only to the income portion above $87,850. Also note that this is only the federal tax. Your state will have additional taxes as well. Aside from those percentages, there are a few other flat taxes, but they are considerably smaller in magnitude.
  • The money you donate to a nonprofit (aka. 501(c)(3)) organization can be subtracted from your taxable income. This means that you will most likely get a refund when you file your taxes. Why? Because when you fill out your W4 form, you'll basically tell your employer how much money to withhold from your paycheck for tax purposes. If you don't account for your future donations, more money will be withheld than is appropriate and the discrepancy will be paid back to you after you file your taxes. Ideally, you want to take your donations into account and fill out the W4 form such that there are no discrepancies. That means you'll get your money now rather than later. (I haven't gone through this process myself, so there is some uncertainty here.)
  • You can claim tax deduction for up to 50% of your wages. That means if you make a lot of money in one year, even if you donate most of it, you'll be able to reduce your taxable income by a maximum of 50%. The rest goes over to the next year.
  • When RSUs vest, their value is treated as ordinary income for tax purposes. When you sell them, the difference is taxed as a capital gain (or loss).
  • Stock options have a more complicated set of tax rules, and you should understand them if you are considering a company that offers them.
  • You can't have your employer donate money or stock for you to bypass the taxes. I've asked.

 

Calculating donations

To calculate exactly how much I could donate if I worked at a given company, I've created this spreadsheet. (This is an example with completely fictitious company offers with very low numbers, but the calculations should be correct.) Let me walk you through the spreadsheet.

 

Time discounting (Cell B1)

Money now is more valuable than money later. By how much? That's a very complicated question. If you invest your money now, you might be able to make something like 10% annually with some risk.[3] If you are donating to a charity, and they are growing very rapidly, then they can do a lot with your money right now, and you should account for that as well. If you expect the charity to double in size/effectiveness/output in the next year, then you might use a discount rate as high as 50%. I chose to use 20% annual discount rate based on my own estimates. Since I'm doing monthly compounding, the spreadsheet value is slightly higher (~22%). You can look at the column K to see how the future value of a dollar is being discounted. Note, for example, that a dollar in 12 months is worth 80¢ to me now. This discounting rate is especially important to keep in mind when examining startups, because almost all their compensation lies in the future. The further away it is, the more heavily you have to discount it.

 

Cost of living (Cell B2)

This is how much pre-tax money a year I'm not going to donate. See column L for the monthly expenses. We time-discount those dollars as well.

 

Offers (Cells A4-I15)

This is where you plug-in the offers you get. Bonus row is for cash bonus. Equity row is for the total equity the company offers you. I use the dollar amount, but you'll notice that for some of them I'm computing the dollar amount as: RSUs the company is giving me * current share price. For private companies, this is value I expect my equity to have when the company goes public. For Square it looks like: (percent of the company I'll own) * (my guess at valuation of the company at IPO) - (cost to exercise my options). For Twitter it looks like: (growth factor up to IPO) * (current price per share) * (RSUs I am granted). (Again, the numbers are completely made up.) In my calculations I'm not expecting public companies' share price to rise or fall. If you disagree, you should adjust for that as well.

 

Monthly projections (Cells A18-I66)

We are going to look at how much money we'll be making per month for the next four years. (Four years because our equity should be fully vested by that time.) If you are certain that you will stay at the company for less time than that, then you should consider a shorter timeline. This might affect companies differently. For example, most of the equity you get at Amazon comes during the last two years. If you are not going to be there, you are missing out on a big part of your offer.

For companies that I was seriously considering, I created two columns: one for cash wages and one for equity wages. This way I can do taxes on them more precisely.

Let's go through the Google's offer:

  • For the first year we'll be only making our standard salary.

  • After the first year, we get our cash bonus (green font). Here we are assuming it'll be 15% of our salary. We also get 25% of our RSUs vested (salmon background).

  • For the remainder of the second year, we are making our normal salary. Each month we also get 1/48th of our original equity offer.

  • Google also has an equity bonus system, where each year you can get a bonus of up to 50% of your original equity offer. This bonus will be paid in RSUs, and it vests over 4 years, but with no cliff. So we count that as well, but I'm assuming I'm only going to get 15%, not the full 50%.

  • In year 3 everything is basically the same, except now we got our second equity bonus, so we have two of them running simultaneously.

  • In year 4, we have three of them running simultaneously.

For pre-IPO companies, I've estimated when they'll go IPO. Most have clauses in place that don't allow you to sell your shares until after half a year or so after the IPO. I'm assuming I will sell/donate all my shares then, and then continue selling/donating them as they continue vesting.

 

Sum (Cells A68-I71)

In row 68 we have the total sum. This is the amount of pre-tax dollars we expect to earn in the next four years (remember that this amount has been adjusted for time-discounting, so it'll seem much lower than you'd normally expect). L68 is how much money we are spending on ourselves during those four years.

In row 69 we subtract our living expenses to get the amount of money we'll be able to donate. Note that I'm subtracting it from the cash column, leaving the equity column alone (for the companies where I split the two).

In row 70 we account for taxes. Note that our living expenses already accounted for the taxes we pay up to $65k, so the rest of it will be taxed at around 28% or higher. You could sell your shares, or you could just donate your shares directly to your charity. (That's what we are doing with our Google offer.)

In row 71 we simply sum up the donations from cash and equity.

 

Disclaimer 1: while I tried as hard as I could to double check this spreadsheet, there might still be mistakes there, so use it with caution and triple check everything. The tax calculations as they are right now are wrong, and you'll have to redo them (basically the whole Row 70) based on your own numbers.

Disclaimer 2: this spreadsheet is not great for evaluating an offer from a startup, since it doesn't capture the associated uncertainty and risk. Furthermore, if you expect the startup to succeed after more than 4 years, to correctly compare it to other companies you'll have to compute more than 48 months and potentially start accounting for things like promotions and raises.

 

Picking the one

All right, so how do you actually pick the best company? It's not as simple as picking the one with the highest EV, since you have to account for risk involved with startups and even pre-IPO companies. In fact, you should be surprised if your offers from public companies have a higher EV than offers from startups. If that's the case, I'd double check your calculations.

This is where it becomes extremely crucial to narrow down your uncertainty. When is the company going to IPO? What is the likely valuation? Does the company have a lot of competitors? Does the company have the necessary talent to execute on their plan? What's the company's history? What is the employee churn rate (especially for executives)? How well is the company doing financially? Who are the investors? Etc, etc, etc... There is a ton of questions you should be asking, and you should be asking them to everyone whose opinion on this issue you can respect. Honest opinion from an informed and knowledgeable neutral party is worth a LOT here!

You should also talk to the people at the company. Your recruiter will connect you to the right people if you ask. Keep in mind that nobody there will tell you that the company is going to go bankrupt or fail. But you can still get some valuable estimates, and then potentially discount them down a bit. You can even ask for their opinion on other companies you are interviewing with. Expect them to completely throw the other company under the bus though, but even so, you could get a lot of valuable criticism and bring it up when you talk to that other company. Overall, expect a lot of conflicting messages.

Keep in mind the charities you'll be donating to. What kind of donors do they have already? Are most people donating a bit from their salary? In that case, a more risky venture might be reasonable. Can they really use some money right now, or would they be a lot more effective later on with a large capital? What's their time discount rate? If you care about your charity, you can help them diversify their donor pool.

For me, it was a hard choice between big public companies (primary candidate: Google) and close to IPO companies (primary candidates: Twitter and Square).

 

Negotiating

You have to negotiate your offer. You have to have to have to HAVE TO. For any given company, you'll be able to get them to up their offer at least once and potentially thrice. Example: Google upped my offer three times.

  • Some companies will tell you their offer is not negotiable. That's not true.
  • It's much easier to leverage similar companies against each other. Leverage big public companies against each other; leverage pre-IPO companies against each other; etc... Leveraging between those categories is a bit more difficult, because startups know they can't compete with the raw cash value you are offered at bigger companies. The only thing they can do is up their equity offer and hope that they are a much better personal fit for you than the large companies.
  • Recruiters will ask you very directly what the other companies are offering you. You can choose to disclose or not to disclose. If you don't disclose, the company will come back to you with their standard offer. That offer might be higher or lower than you expected. (Example: The first offer I got from Google was significantly worse than initial offers I got from Facebook and Amazon.) If you tell them what offers you have (and you should only disclose details of your very best offers), then they'll very likely match or come in a bit stronger. Usually you don't have much to gain by disclosing your other offers upfront. You can always do so later. However, you should let your recruiters know that other companies did make an offer, or you are expecting them to. That gives you more leveraging power.
  • Sign-on bonus is very easy to negotiate. You can easily convince a company to match a sign-on bonus their competitor has offered.
  • Negotiating salary is much harder, but, again, usually you can convince a company to match a salary their competitor has offered or at least come closer to it. If you are interviewing with startups, their salary offer will usually be lower than at bigger companies and even harder to negotiate. ("Cash is king" is the common phrase used there.)

First negotiating phase: simply email / call back your recruiter (who is now your best friend, right?) and tell them that the offer is somewhat lower than you expected, you have other better offers from other companies, and you are wondering if they can increase their offer. If the company made you a clearly worse offer than another similar company, you should be very open about it.

Second negotiating phase: matching other companies. This is when it makes the most sense to disclose your other offers. For example, I used my Amazon and Facebook offers to convince Google to up their offer significantly. For some reason their original offer was very low, but seeing their competitors with much better offers convinced them to update pretty quickly. You can also bring up the perks one company has that the other doesn't (e.g. donation matching or unlimited PTO). The company can make up for that with salary/equity. There is some difficulty in using offers from private companies as leverage, because there is not much information you can disclose about them. You can talk about the number of shares you'll have, but it might not mean anything to the other recruiters if they are not familiar with the startup.

Third negotiating phase: once you picked the company you'll work for, go back to them and say something along the lines of "I really like the offer and the company, but there are a few things that don't make it ideal for me. One of your competitors did this, and another company has that. Right now I'm inclined to go with your competitor, but it's a tough decision, and I would rather go with you. I think if you can make me an offer with the following parameters, it'll make my decision extremely easy, and I'll sign on the spot." Include offer letters from other companies, especially the ones that have them beat or beat on some parameters. Notice the key promise at the end: you will sign with them. Your recruiter will have a lot more leverage in fighting for you if you make that promise. You are not legally obligated to follow through with your promise, but I wouldn't advise breaking it or using it just to extract more value to use as leverage against other companies. Use this tactic at the very end to extract that last bit of value from the company that's already the best. This is what I did with Google. I asked for about 3% higher salary and 12% more equity than what they were offering, and they came back with the exact numbers I requested, which means I should have asked for more. My advice would be to ask for about twice or may be even three times as much (6% and 30% respectively). Even if they come back with a compromise, it'll very likely be more than 3% and 12% increase. If not, you can try to barter one more time.

I'm sure some people will cringe at this kind of haggling, but, in all honesty, this is what recruiters expect, and they are very much used to it. Nobody even blinked an eye when I started negotiating, even on second and third rounds. However, some recruiters might try to make you feel guilty. They'll say that if you really want to work at their startup, then you shouldn't really care about your compensation. Most points they'll make will even be valid, but if you are trying to optimize for donations, then you have to make the compensation the most important factor in your decision. I've actually told most of my recruiters that I plan to donate most of my salary to charities. I don't think that got me higher offers, but it made me come off less like a greedy jerk.

At the end of the day, the company wants you, but they want to pay you as little as possible. But, given the choice of having you and paying you the most you deserve VS. not having you, all companies will pick the first option. ALL OF THEM. This is one of the best perks of being a talented software engineer in the bay area.

Once you accept the offer, don't forget to email everyone else and let them know. Thank everyone that helped you. Some recruiters will be surprised by your decision, and some will even fight really hard to get you to reconsider.

 

 


 

[1] None of the interviews required a data structure more complicated than a heap. All the answers had a very easy to compute complexity, either polynomial, polynomial * logarithmic, or factorial. The most weird one was probably O(√n) for computing prime numbers.

[2] Some problems I did during actual single-round match-up (SRM) competitions, which is good for training yourself how to code and think faster than you are used to. I also did a lot of old SRM problems, which have solutions and explanations posted in case I couldn't get them. I could easily do problem 1 & 2 in the easy division, and could do problem 3 most of the time. I didn't really bother with the hard division, and none of the interview questions were ever as hard as problem 3 in the easy division.

[3] According to the comments, this number is too high. Pick your own best estimate.

Estimates vs. head-to-head comparisons

12 paulfchristiano 04 May 2013 06:45AM

(Cross-posted from my blog.)

Summary: when choosing between two options, it’s not always optimal to estimate the value of each option and then pick the better one.

Suppose I am choosing between two actions, X and Y. One way to make my decision is to predict what will happen if I do X and predict what will happen if I do Y, and then pick the option which leads to the outcome that I prefer.

My predictions may be both vague and error-prone, and my value judgments might be very hard or nearly arbitrary. But it seems like I ultimately must make some predictions, and must decide how valuable the different outcomes are. So if I have to evaluate N options, I could do it by evaluating the goodness of each option, and then simply picking the option with the highest value. Right?

continue reading »

View more: Next