Maximizing Your Donations via a Job

108 Alexei 05 May 2013 11:19PM

In November of 2012 I set a goal for myself: find the most x-risk reducing role I can fill. At first I thought it would be by working directly with MIRI, but after a while it became clear that I could contribute more by simply donating. So my goal became: find the highest paying job, so I can donate lots of money to CFAR and MIRI.

A little bit of background on me. Started programming in 2000. Graduated in 2009 with Bachelor's in computer science. Worked for about a year and a half at a game company. Then did my own game startup for about a year. Then moved to the bay area and joined a game startup here, which was acquired 10 months later. Worked a bit at the new company and then left. So, just under four years of professional programming experience, but primarily in the game industry. Almost no leadership / managerial experience, aside from the startup I did where I hired freelancers.

Below is my experience of finding a software engineering job in the Silicon Valley. If you are not an engineer or not in the Silicon Valley, I think you'll still find a lot of useful information here.

 

Pre-game

Before sending out my resume, I spent about a month preparing. I read Intro to Algorithms, which was very good overall, but not a huge help in preparing for interviews.[1] I read Cracking the Coding Interview, which was extremely helpful. (If you read only one book to prepare, make it this one.) The book has a lot of questions that are similar to the ones you'll actually see during interviews. I also did TopCoder problems, which were pretty helpful as well.[2] Looking back, I wish I spent more time finding actual interview questions online and doing more of those (that's why CCI book was so helpful).

After several weeks of preparation, I compiled a long list of companies I was going to apply to. I checked on GlassDoor to see what kind of salary I could expect at each one. I then rated all the companies. Companies with low salaries and poor personal fit received the lowest rating.

I started by applying to companies with the lowest ratings. This way I could use them as practice for the companies I thought would actually make a competitive offer. This was the right move and worked very well. (Another friend of mine did the same approach with good results as well.) Remember, you are not just doing those interviews to practice the coding problems, you are practicing pitching yourself as well.

 

Interviewing with a company

Standard procedure for applying to a tech company:

1. Send them your resume.

  • Proofread your resume. Let your friends proofread it.
  • Make sure there are only relevant things on it. When I applied to tech companies, I removed a lot of game-specific things from my resume. When I applied to companies that did 3D graphics, I made sure I had all my 3D graphics experience listed. I ended up with two version of my resume.
  • Have your resume in DOC, PDF, and TXT formats. This way you'll always have the right one when you upload / paste it.
  • For a few companies, I had a friend or friend of a friend who referred me. This REALLY HELPS in two ways: 1) your resume will be processed a lot faster, 2) if your friend is a great engineer/employee, you'll be taken a lot more seriously, and the company will fight for you a lot harder.

2. You'll get an email from the recruiter and setup a time to speak, where you'll talk about yourself, what you've done, why you are interested in their company, and so on. You can and should ask them questions as well.

  • When you start getting multiple calls each day, make sure you know who is calling. There is nothing worse than talking about the challenges of streaming music to a car sharing startup. (True story.)
  • Read about the company on Wikipedia before the call. Know the basic stuff. Look at their website and read the About page.
  • Find the thing that makes the company special and successful. Find the thing that you actually think is cool about the company. Those are your answers for why you want to work there.
  • Ask non-technical questions: How is the company structured? How many teams are there? How many employees? Engineers? Think of other intelligent questions to ask.
  • In my experience, it's not very beneficial to tell them you are interviewing with a dozen other companies. When they ask who else you are interviewing with, just name a few companies, especially the competitors / similar companies.
  • Be SUPER NICE to your recruiter. They are your main point of contact with the company. They'll be the one fighting to get you the best offer.

3. You'll have a technical phone interview with a software engineer where you'll solve a problem or two on collabedit or some similar website. At the end, you'll get a few minutes to ask them questions too.

  • All the usual interviewing tips apply here. E.g. talk out loud, your interviewer doesn't know what you are thinking.
  • Most companies don't care what language you use, as long as it's mainstream. (I used C# for almost all my coding questions.)
  • DO NOT start answering the question by writing code. If the questions seems vague, ask about the context. Who'll be using this solution? Definitely ask about the kind of data you are working with. If it's integers, are they random? Over some small range or over all possible integers?
  • List out metrics for various approaches: brute-force solution, optimized for speed solution, optimized for memory solution. Here is a question I saw a few times: Write a data structure which can accept and store integers, and can check if there exist two integers that sum up to a given number. There are multiple solutions, and the best one depends on the ratio of addInteger to checkForSum calls.
  • The previous steps should only take you a minute or two. Once you've decided what the best approach is, then you can write the solution. When you are done, check for errors, then run through several examples. Do a simple example and a slightly complicated example. When you find a bug, don't be hasty in fixing it. Understand why it happened and make sure you won't introduce new bugs by fixing it.
  • If everything works, make sure you handle errors correctly. Can you handle invalid input? Input that violates your assumptions? (As a reminder, I leave “\\Check for errors” comments in appropriate spots as I code the solution.)
  • When you are done, ask the interviewer questions. Ask them to tell you about what they do, if they haven't already. What have they been working on recently? What technologies/languages do they use at the company? Do they use Scrum/Agile? Pair-programming? Come up with other intelligent questions to ask.

 

4. You'll be invited for an on-site interview which will be 3-6 hours long, at least half of which will be coding on a white-board. (Although, a friend told me he brought his laptop with him, and most people were fine with him coding on it.)

  • All the previous tips apply.
  • Be on time. Take bathroom breaks when you need them. I found that drinking water during the interview keeps me refreshed. Remember your posture, body-language, and eye-contact skills.
  • Learn how to talk out loud as you are writing out your solution. If you are stuck, explain what you are thinking, and what your intuition is telling you.
  • Learn how to read your interviewers. If you say, "Here we should check for the null case or for empty array," and they go "Yeah, yeah, okay," they are not the type of interviewer that really cares about error conditions, so you can be somewhat more lax there. By the time I was finishing my on-site interviews, I could tell if my solution was right just by the interviewer's body language.
  • When you are done, ask them questions. What are they working on? What's the thing they like most about the company? What's their least favorite thing about the company? (Another way to phrase that: What's one thing you would change if you could change anything about the company?) Do they have to work overtime? How are the people here? Can you switch between projects? Are there company wide events? In all my interviews I've never met an interviewer that didn't try to sell their company really hard. People will always tell you their company is the best place to work.
  • If the person is a manager or a director, ask them higher level questions. What kind of culture are they trying to create? What are the current big challenges? Where do they want the company to be in the next 5 years? How does one advance in the company? (Usually there is a managerial and a technical track.) How often are reviews done? How are they structured?

 

5. You'll get a call from the recruiter congratulating you on an offer. They'll go over the offer details with you.

  • Before they make you an offer, they'll check if you are actually seriously considering their company. If you told a startup you are also interviewing with Google, they might suspect that you are not seriously considering them. Unless you dissuade those fears, they might actually not even make you an offer. (Happened to me with Rdio.)
  • If you didn't get an offer, try to get as much info as you can. What happened? What can you improve on? Below are the reasons why I didn't get an offer after an on-site interview:
    • Not doing well on a technical question. (Happened twice; one time because of a very obnoxious interviewer.)
    • Not interviewing for quite the right position (that on-site interview ended early).
    • Not having the necessary experience (a lot more important to startups than bigger companies).
    • Not being passionate enough about the company.
  • If this is not a good timing for the offer, e.g. it's one of your first interviews, then tell them so. They will probably wait to give you the offer details until you are ready to consider it.
  • The recruiter will likely ask what's important to you in an offer. How are you going to make your decision? What I've said is that compensation will be an important factor in my decision, but that the team/project/etc. are important considerations as well.

 

6. You have a few days (usually around 5 business days) until the offer expires to decide if you want to accept it.

  • Sometimes the offer will expire before you've received offers from other companies. This is why it's important to interview in rough order of ranking, so that you can just let those offers go, knowing you'll have much better ones soon. If you want to hold on to the offer, just ask your recruiter for an extension. It'll be much easier to get an extension at big companies, especially if you are interviewing for a generic position.
  • If you decline the offer, let them know.

 

Always be very nice, friendly, and polite. Walk the fine line between telling the truth and saying the right thing. Ideally, make sure those are the same. Even if you are interviewing with a company you have no intention of working at, make sure to find something you really like about them, something that makes them stand out to you. Always have a good answer to: "Why do you want to work here?"

Before each on-site interview make sure you research the company thoroughly. Use their product. Think of ways to improve it. It's very helpful if you can meet with someone that works there and talk to them. See if they can give you any tips on the interview process. Some companies (e.g. AirBnB) want people that are extremely passionate about their product. Some companies focus more than usual on architectural questions. Many companies expect the engineers to have some familiarity with UI/UX and the ability to think about a feature from all angles.

 

Managing your time

I sent my resume to 78 companies, had at least a phone conversation with a recruiter with 27 of them, had an on-site interview with 16 companies, and received 12 offers. Out of those, I've only seriously considered 3. (Companies with lower ratings had an atrocious response rate.)

My time-line ended up looking something like this:

  • Week 1: Started applying to low-rated companies. About 2 phone interviews.
  • Week 2: About 7 phone interviews. One on-site interview. Sending out more resumes.
  • Week 3: About 3 phone interviews.
  • Week 4: About 15 phone interviews. A few meetings with friends of friends, who ended up referring me. 1 on-site interview. Sent my resume to all the high-rated companies. (During this week interviewing became a full-time job.)
  • Week 5: About 10 phone interviews. 4 on-site interviews.
  • Week 6: 8 phone interviews. 4 on-site interviews.
  • Week 7: 4 phone calls. 5 on-site interviews.
  • Week 8: 12 phone calls. 2 on-site interviews.
  • Week 9: About 8 calls a day for a few days, while I negotiated with my top companies.
  • (These are strictly lower bounds for phone calls. On-site data is pretty accurate.)

Some companies move fast, some companies move slow. Google took 2 weeks from the on-site interview to the offer call. This is very common for them, but most other companies move faster. With Amazon, I actually interviewed with two different branches. With one branch things were going well, until they dropped the ball and never got back to me, even after I pestered them. This is unusual; although, Twitter did something similar, but then ended up responding with an on-site invitation. With the other Amazon branch, when I got home from the on-site interview, I already had an email saying they were going to make an offer. This is extremely fast. (I had a very good reference for that position.) Most companies take about a week between on-site and offer. The whole process, from first call to offer, takes about three weeks.

If your recruiter doesn't respond to you during 4 days or longer, shoot them an email. They might have forgotten to respond, or thought they did, or may be things are moving slowly, or may be they decided not to pursue. You want to be clear on where you stand with all the companies you are applying to.

The timing is pretty important here. You want your top-rated companies to give you an offer within a span of a week. This way you'll be able to leverage all those offers against each other.

If your current job position is already almost optimal for your goals, then it's possible you can do a few interviews, get a few offers and pick the best one, which will give you some marginal improvement. Or use those offers to leverage a raise at your existing company. But if you are pretty sure your current job has not been optimized for your goal, then I'd say, contrary to popular wisdom, just leave and spend a full month interviewing. (Or, even better, if you can, take a long "vacation".) You just can't do this kind of intense interviewing while holding another job. The one exception to this rule I can think of is if one of your highest-rated companies is a competitor with your current employer. Then you can leverage that!

Value of information is extremely high during this process. Talk to all the companies you can, talk to all the people you can. Once you have the final list of companies you are considering, reduce your uncertainty on everything. Validate all your assumptions. (Example: I was sure Google matched donations up to $12k, but turns out it's only up to $6k.)

 

How to evaluate your offer

There are 4 basic components in an offer: sign-on bonus, base salary, equity, and bonus.

Sign-on bonus. Most companies will be okay offering something like $12k sign-on bonus. Some will offer more. Most startups probably won't offer any.

Base salary. This is pretty consistent across most companies. Based on your experience, you'll be given a title (e.g. Senior Software Engineer or SE 2), and that title will determine the range of the salary you can expect. If you are good, you can demand a salary at the top of that range, but it's extremely hard to go higher.

Equity. This is the most interesting part. A good amount of value will come from this portion. With a startup, it'll be most of it. Here are two things to pay attention to:

  • Is the company public or private? If it's public, you are most likely going to be given RSUs (restricted stock units), which will basically convert to normal company shares when they vest. For private companies, see the section below.
  • What's the vesting schedule? For almost all companies you'll get 25% of your shares right after your first year. (This is called a 'cliff'.) After that you'll be given the appropriate fraction either monthly (e.g. at Google) or quarterly (e.g. at Facebook). Amazon is an example of a company where the vesting schedule is somewhat different: 5% after year 1, 15% after year 2, and then 20% each semester for the next two years.

Bonus. This is the bonus system the company has setup. You can't negotiate it, but it's important to take it into account.

  • There will usually be a cash bonus that's based on your salary. It'll have a target percent (e.g. 15%). If you can find out how many people hit their target, that will be very helpful. However, most companies don't share or simply don't have that information.
  • Some companies also have equity bonuses. Try to get as much info on those as you can. Don't assume that you'll get the maximum bonus even if you work hard. If you have friends working at that company, ask them what kind of bonuses they've been getting.
  • Lots of startups don't have bonus systems in place.

Other factors.

  • Donation matching: Google matches up to $6k (you donate $6k to any charity, they'll donate another $6k). Craigslist matches 3:1 up to 10% of your salary. Most companies don't have anything like that, and you can't negotiate it.
  • Paid Time Off: Google offers 2 weeks, all other companies I was considering offer 3 weeks, and some even have unlimited PTO. This is not negotiable in most companies.
  • Commute: how far will you have to travel to work? Are you okay moving closer to work? (Google and Facebook have shuttles that can pick you up almost anywhere, so you could work while you commute.)
  • People/culture/community/team/project are all important factors as well, depending on what you want. If you are going to spend the next several years working on something, you should be building up skills that will still be valuable in the future.

 

Thinking about private companies

If the company is private, you might be given RSUs or you might be given stock options. With stock options, you'll have to pay the strike price to exercise your options. So the total value your options have is: (price of a share - strike price) * number of shares.

You can't do anything with your shares until the company gets acquired or goes public. Some companies have liquidation events, but those are pretty rare. Most companies don't have them, and the ones that do only extend the opportunity to people that have been with the company for a while. There are also second-hand markets, but I don't know much about those.

If you are completely risk-intolerant, then just go with a public company, and don't consider private companies. (This is actually not exactly true. Just because a company is public, doesn't mean its risk-free, and just because a company is private doesn't mean there is a lot of risk. There are other important factors like the size of the company, their market diversity, and how long they've been around.) If you are okay with some risk, then you want a company that's close to an IPO or is likely to get acquired soon. If you want to have a chance to make more than a few million dollars, either start your own company or join a very early stage startup (my top pick would be Ripple). Before doing so, check out the stats on startups to make sure you understand how likely any given startup is to fail and make sure you understand the concepts of inside/outside view.

 

Taxes

It's crucial to understand all the tax implications of your salary, equity, and donations. I'm not going to go into all the details, there are a lot of resources out there for this, but you should definitely read them until it's crystal clear how you will be taxed. I'll highlight a few points:

  • Understand the tax rate schedule and notice the new 39.6% tax bracket. If your income is $100k, that doesn't mean you get taxed 28% on all of it. 28% applies only to the income portion above $87,850. Also note that this is only the federal tax. Your state will have additional taxes as well. Aside from those percentages, there are a few other flat taxes, but they are considerably smaller in magnitude.
  • The money you donate to a nonprofit (aka. 501(c)(3)) organization can be subtracted from your taxable income. This means that you will most likely get a refund when you file your taxes. Why? Because when you fill out your W4 form, you'll basically tell your employer how much money to withhold from your paycheck for tax purposes. If you don't account for your future donations, more money will be withheld than is appropriate and the discrepancy will be paid back to you after you file your taxes. Ideally, you want to take your donations into account and fill out the W4 form such that there are no discrepancies. That means you'll get your money now rather than later. (I haven't gone through this process myself, so there is some uncertainty here.)
  • You can claim tax deduction for up to 50% of your wages. That means if you make a lot of money in one year, even if you donate most of it, you'll be able to reduce your taxable income by a maximum of 50%. The rest goes over to the next year.
  • When RSUs vest, their value is treated as ordinary income for tax purposes. When you sell them, the difference is taxed as a capital gain (or loss).
  • Stock options have a more complicated set of tax rules, and you should understand them if you are considering a company that offers them.
  • You can't have your employer donate money or stock for you to bypass the taxes. I've asked.

 

Calculating donations

To calculate exactly how much I could donate if I worked at a given company, I've created this spreadsheet. (This is an example with completely fictitious company offers with very low numbers, but the calculations should be correct.) Let me walk you through the spreadsheet.

 

Time discounting (Cell B1)

Money now is more valuable than money later. By how much? That's a very complicated question. If you invest your money now, you might be able to make something like 10% annually with some risk.[3] If you are donating to a charity, and they are growing very rapidly, then they can do a lot with your money right now, and you should account for that as well. If you expect the charity to double in size/effectiveness/output in the next year, then you might use a discount rate as high as 50%. I chose to use 20% annual discount rate based on my own estimates. Since I'm doing monthly compounding, the spreadsheet value is slightly higher (~22%). You can look at the column K to see how the future value of a dollar is being discounted. Note, for example, that a dollar in 12 months is worth 80¢ to me now. This discounting rate is especially important to keep in mind when examining startups, because almost all their compensation lies in the future. The further away it is, the more heavily you have to discount it.

 

Cost of living (Cell B2)

This is how much pre-tax money a year I'm not going to donate. See column L for the monthly expenses. We time-discount those dollars as well.

 

Offers (Cells A4-I15)

This is where you plug-in the offers you get. Bonus row is for cash bonus. Equity row is for the total equity the company offers you. I use the dollar amount, but you'll notice that for some of them I'm computing the dollar amount as: RSUs the company is giving me * current share price. For private companies, this is value I expect my equity to have when the company goes public. For Square it looks like: (percent of the company I'll own) * (my guess at valuation of the company at IPO) - (cost to exercise my options). For Twitter it looks like: (growth factor up to IPO) * (current price per share) * (RSUs I am granted). (Again, the numbers are completely made up.) In my calculations I'm not expecting public companies' share price to rise or fall. If you disagree, you should adjust for that as well.

 

Monthly projections (Cells A18-I66)

We are going to look at how much money we'll be making per month for the next four years. (Four years because our equity should be fully vested by that time.) If you are certain that you will stay at the company for less time than that, then you should consider a shorter timeline. This might affect companies differently. For example, most of the equity you get at Amazon comes during the last two years. If you are not going to be there, you are missing out on a big part of your offer.

For companies that I was seriously considering, I created two columns: one for cash wages and one for equity wages. This way I can do taxes on them more precisely.

Let's go through the Google's offer:

  • For the first year we'll be only making our standard salary.

  • After the first year, we get our cash bonus (green font). Here we are assuming it'll be 15% of our salary. We also get 25% of our RSUs vested (salmon background).

  • For the remainder of the second year, we are making our normal salary. Each month we also get 1/48th of our original equity offer.

  • Google also has an equity bonus system, where each year you can get a bonus of up to 50% of your original equity offer. This bonus will be paid in RSUs, and it vests over 4 years, but with no cliff. So we count that as well, but I'm assuming I'm only going to get 15%, not the full 50%.

  • In year 3 everything is basically the same, except now we got our second equity bonus, so we have two of them running simultaneously.

  • In year 4, we have three of them running simultaneously.

For pre-IPO companies, I've estimated when they'll go IPO. Most have clauses in place that don't allow you to sell your shares until after half a year or so after the IPO. I'm assuming I will sell/donate all my shares then, and then continue selling/donating them as they continue vesting.

 

Sum (Cells A68-I71)

In row 68 we have the total sum. This is the amount of pre-tax dollars we expect to earn in the next four years (remember that this amount has been adjusted for time-discounting, so it'll seem much lower than you'd normally expect). L68 is how much money we are spending on ourselves during those four years.

In row 69 we subtract our living expenses to get the amount of money we'll be able to donate. Note that I'm subtracting it from the cash column, leaving the equity column alone (for the companies where I split the two).

In row 70 we account for taxes. Note that our living expenses already accounted for the taxes we pay up to $65k, so the rest of it will be taxed at around 28% or higher. You could sell your shares, or you could just donate your shares directly to your charity. (That's what we are doing with our Google offer.)

In row 71 we simply sum up the donations from cash and equity.

 

Disclaimer 1: while I tried as hard as I could to double check this spreadsheet, there might still be mistakes there, so use it with caution and triple check everything. The tax calculations as they are right now are wrong, and you'll have to redo them (basically the whole Row 70) based on your own numbers.

Disclaimer 2: this spreadsheet is not great for evaluating an offer from a startup, since it doesn't capture the associated uncertainty and risk. Furthermore, if you expect the startup to succeed after more than 4 years, to correctly compare it to other companies you'll have to compute more than 48 months and potentially start accounting for things like promotions and raises.

 

Picking the one

All right, so how do you actually pick the best company? It's not as simple as picking the one with the highest EV, since you have to account for risk involved with startups and even pre-IPO companies. In fact, you should be surprised if your offers from public companies have a higher EV than offers from startups. If that's the case, I'd double check your calculations.

This is where it becomes extremely crucial to narrow down your uncertainty. When is the company going to IPO? What is the likely valuation? Does the company have a lot of competitors? Does the company have the necessary talent to execute on their plan? What's the company's history? What is the employee churn rate (especially for executives)? How well is the company doing financially? Who are the investors? Etc, etc, etc... There is a ton of questions you should be asking, and you should be asking them to everyone whose opinion on this issue you can respect. Honest opinion from an informed and knowledgeable neutral party is worth a LOT here!

You should also talk to the people at the company. Your recruiter will connect you to the right people if you ask. Keep in mind that nobody there will tell you that the company is going to go bankrupt or fail. But you can still get some valuable estimates, and then potentially discount them down a bit. You can even ask for their opinion on other companies you are interviewing with. Expect them to completely throw the other company under the bus though, but even so, you could get a lot of valuable criticism and bring it up when you talk to that other company. Overall, expect a lot of conflicting messages.

Keep in mind the charities you'll be donating to. What kind of donors do they have already? Are most people donating a bit from their salary? In that case, a more risky venture might be reasonable. Can they really use some money right now, or would they be a lot more effective later on with a large capital? What's their time discount rate? If you care about your charity, you can help them diversify their donor pool.

For me, it was a hard choice between big public companies (primary candidate: Google) and close to IPO companies (primary candidates: Twitter and Square).

 

Negotiating

You have to negotiate your offer. You have to have to have to HAVE TO. For any given company, you'll be able to get them to up their offer at least once and potentially thrice. Example: Google upped my offer three times.

  • Some companies will tell you their offer is not negotiable. That's not true.
  • It's much easier to leverage similar companies against each other. Leverage big public companies against each other; leverage pre-IPO companies against each other; etc... Leveraging between those categories is a bit more difficult, because startups know they can't compete with the raw cash value you are offered at bigger companies. The only thing they can do is up their equity offer and hope that they are a much better personal fit for you than the large companies.
  • Recruiters will ask you very directly what the other companies are offering you. You can choose to disclose or not to disclose. If you don't disclose, the company will come back to you with their standard offer. That offer might be higher or lower than you expected. (Example: The first offer I got from Google was significantly worse than initial offers I got from Facebook and Amazon.) If you tell them what offers you have (and you should only disclose details of your very best offers), then they'll very likely match or come in a bit stronger. Usually you don't have much to gain by disclosing your other offers upfront. You can always do so later. However, you should let your recruiters know that other companies did make an offer, or you are expecting them to. That gives you more leveraging power.
  • Sign-on bonus is very easy to negotiate. You can easily convince a company to match a sign-on bonus their competitor has offered.
  • Negotiating salary is much harder, but, again, usually you can convince a company to match a salary their competitor has offered or at least come closer to it. If you are interviewing with startups, their salary offer will usually be lower than at bigger companies and even harder to negotiate. ("Cash is king" is the common phrase used there.)

First negotiating phase: simply email / call back your recruiter (who is now your best friend, right?) and tell them that the offer is somewhat lower than you expected, you have other better offers from other companies, and you are wondering if they can increase their offer. If the company made you a clearly worse offer than another similar company, you should be very open about it.

Second negotiating phase: matching other companies. This is when it makes the most sense to disclose your other offers. For example, I used my Amazon and Facebook offers to convince Google to up their offer significantly. For some reason their original offer was very low, but seeing their competitors with much better offers convinced them to update pretty quickly. You can also bring up the perks one company has that the other doesn't (e.g. donation matching or unlimited PTO). The company can make up for that with salary/equity. There is some difficulty in using offers from private companies as leverage, because there is not much information you can disclose about them. You can talk about the number of shares you'll have, but it might not mean anything to the other recruiters if they are not familiar with the startup.

Third negotiating phase: once you picked the company you'll work for, go back to them and say something along the lines of "I really like the offer and the company, but there are a few things that don't make it ideal for me. One of your competitors did this, and another company has that. Right now I'm inclined to go with your competitor, but it's a tough decision, and I would rather go with you. I think if you can make me an offer with the following parameters, it'll make my decision extremely easy, and I'll sign on the spot." Include offer letters from other companies, especially the ones that have them beat or beat on some parameters. Notice the key promise at the end: you will sign with them. Your recruiter will have a lot more leverage in fighting for you if you make that promise. You are not legally obligated to follow through with your promise, but I wouldn't advise breaking it or using it just to extract more value to use as leverage against other companies. Use this tactic at the very end to extract that last bit of value from the company that's already the best. This is what I did with Google. I asked for about 3% higher salary and 12% more equity than what they were offering, and they came back with the exact numbers I requested, which means I should have asked for more. My advice would be to ask for about twice or may be even three times as much (6% and 30% respectively). Even if they come back with a compromise, it'll very likely be more than 3% and 12% increase. If not, you can try to barter one more time.

I'm sure some people will cringe at this kind of haggling, but, in all honesty, this is what recruiters expect, and they are very much used to it. Nobody even blinked an eye when I started negotiating, even on second and third rounds. However, some recruiters might try to make you feel guilty. They'll say that if you really want to work at their startup, then you shouldn't really care about your compensation. Most points they'll make will even be valid, but if you are trying to optimize for donations, then you have to make the compensation the most important factor in your decision. I've actually told most of my recruiters that I plan to donate most of my salary to charities. I don't think that got me higher offers, but it made me come off less like a greedy jerk.

At the end of the day, the company wants you, but they want to pay you as little as possible. But, given the choice of having you and paying you the most you deserve VS. not having you, all companies will pick the first option. ALL OF THEM. This is one of the best perks of being a talented software engineer in the bay area.

Once you accept the offer, don't forget to email everyone else and let them know. Thank everyone that helped you. Some recruiters will be surprised by your decision, and some will even fight really hard to get you to reconsider.

 

 


 

[1] None of the interviews required a data structure more complicated than a heap. All the answers had a very easy to compute complexity, either polynomial, polynomial * logarithmic, or factorial. The most weird one was probably O(√n) for computing prime numbers.

[2] Some problems I did during actual single-round match-up (SRM) competitions, which is good for training yourself how to code and think faster than you are used to. I also did a lot of old SRM problems, which have solutions and explanations posted in case I couldn't get them. I could easily do problem 1 & 2 in the easy division, and could do problem 3 most of the time. I didn't really bother with the hard division, and none of the interview questions were ever as hard as problem 3 in the easy division.

[3] According to the comments, this number is too high. Pick your own best estimate.

Consistence of reciprocity?

0 yttrium 16 December 2012 07:08PM

Many people see themselves in various groups (member of the population of their home country, or their social network), and feel justified in caring more about the well-being of people in this group than about that of others. They will argue with reciprocity: "Those people pay taxes in our country, they are entitled to more support from 'us' than others!" My question is: Is this inconsistent with some rationality axioms that seem obvious? What often-adopted or reasonable axioms are there that make this inconsistent?

More Cryonics Probability Estimates

20 jkaufman 17 December 2012 08:59PM

There are a lot of steps that all need to go correctly for cryonics to work. People who had gone through the potential problems, assigning probabilities, had come up with odds of success between 1:4 and 1:435. About a year ago I went through and collected estimates, finding other people's and making my own. I've been maintaining these in a googledoc.

Yesterday, on the bus back from the NYC mega-meetup with a group of people from the Cambridge LessWrong meetup, I got more people to give estimates for these probabilities. We started with my potential problems, I explained the model and how independence works in it [1]. For each question everyone decided on their own answer and then we went around and shared our answers (to reduce anchoring). Because there's still going to be some people adjusting to others based on their answers I tried to randomize the order in which I asked people their estimates. My notes are here. [2]

The questions were:

  • You die suddenly or in a circumstance where you would not be able to be frozen in time.
  • You die of something where the brain is degraded at death.
  • You die in a hospital that refuses access to you by the cryonics people.
  • After death your relatives reject your wishes and don't let the cryonics people freeze you.
  • Some law is passed that prohibits cryonics before you die.
  • The cryonics people make a mistake in freezing you.
  • Not all of what makes you you is encoded in the physical state of the brain (or whatever you would have preserved).
  • The current cryonics process is insufficient to preserve everything (even when perfectly executed).
  • All people die (existential risks).
  • Society falls apart (global catastrophic non-existential risks).
  • Some time after you die cryonics is outlawed.
  • All cryonics companies go out of business.
  • The cryonics company you chose goes out of business.
  • Your cryonics company screws something up and you are defrosted.
  • It is impossible to extract all the information preserved in the frozen brain.
  • The technology is never developed to extract the information.
  • No one is interested in your brain's information.
  • It is too expensive to extract your brain's information.
  • Reviving people in simulation is impossible.
  • The technology is never developed to run people in simulation.
  • Running people in simulation is outlawed.
  • No one is interested running you in simulation.
  • It is too expensive to run you in simulation.
  • Other.

To see people's detailed responses have a look at the googledoc, but bottom line numbers were:

person chance of failure odds of success
Kelly 35% 1:2
Jim 80% 1:5
Mick 89% 1:9
Julia 96% 1:23
Ben 98% 1:44
Jeff 100% 1:1500

(These are all rounded, but one of the two should have enough resolution for each person.)

The most significant way my estimate differs from others turned out to be for "the current cryonics process is insufficient to preserve everything". On that question alone we have:

person chance of failure
Kelly 0%
Jim 35%
Mick 15%
Julia 60%
Ben 33%
Jeff 95%

 

My estimate for this used to be more positive, but it was significantly brought down by reading this lesswrong comment:

Let me give you a fuller view: I am a neuroscientist, and I specialize in the biochemistry/biophysics of the synapse (and interactions with ER and mitochondria there). I also work on membranes and the effect on lipid composition in the opposing leaflets for all the organelles involved.

Looking at what happens during cryonics, I do not see any physically possible way this damage could ever be repaired. Reading the structure and "downloading it" is impossible, since many aspects of synaptic strength and connectivity are irretrievably lost as soon as the synaptic membrane gets distorted. You can't simply replace unfolded proteins, since their relative position and concentration (and modification, and current status in several different signalling pathways) determines what happens to the signals that go through that synapse; you would have to replace them manually, which is a) impossible to do without destroying surrounding membrane, and b) would take thousands of years at best, even if you assume maximally efficient robots doing it (during which period molecular drift would undo the previous work).

Etc, etc. I can't even begin to cover complications I see as soon as I look at what's happening here. I'm all for life extension, I just don't think cryonics is a viable way to accomplish it.

In the responses to their comment they go into more detail.

Should I be giving this information this much weight? "many aspects of synaptic strength and connectivity are irretrievably lost as soon as the synaptic membrane gets distorted" seems critical.

Other questions on which I was substantially more pessimistic than others were "all cryonics companies go out of business", "the technology is never developed to extract the information", "no one is interested in your brain's information", and "it is too expensive to extract your brain's information".

I also posted this on my blog


[1] Specifically, each question is asking you "the chance that X happens and this keeps you from being revived, assuming that all of the previous steps all succeeded". So if both A and B would keep you from being successfully revived, and I ask them in that order, but you think they're basically the same question, then A basically only A gets a probability while B gets 0 or close to it (because B is technically "B given not-A")./p>

 

[2] For some reason I was writing ".000000001" when people said "impossible". For the purposes of this model '0' is fine, and that's what I put on the googledoc.

A definition of wireheading

35 Anja 27 November 2012 07:31PM

Wireheading has been debated on Less Wrong over and over and over again, and people's opinions seem to be grounded in strong intuitions. I could not find any consistent definition around, so I wonder how much of the debate is over the sound of falling trees. This article is an attempt to get closer to a definition that captures people's intuitions and eliminates confusion. 

Typical Examples

Let's start with describing the typical exemplars of the category "Wireheading" that come to mind.

  • Stimulation of the brain via electrodes. Picture a rat in a sterile metal laboratory cage, electrodes attached to its tiny head, monotonically pushing a lever with its feet once every 5 seconds. In the 1950s Peter Milner and James Olds discovered that electrical currents, applied to the nucleus accumbens, incentivized rodents to seek repetitive stimulation to the point where they starved to death.  
  • Humans on drugs. Often mentioned in the context of wireheading is heroin addiction. An even better example is the drug soma in Huxley's novel "Brave new world": Whenever the protagonists feel bad, they can swallow a harmless pill and enjoy "the warm, the richly coloured, the infinitely friendly world of soma-holiday. How kind, how good-looking, how delightfully amusing every one was!"
  • The experience machine. In 1974 the philosopher Robert Nozick created a thought experiment about a machine you can step into that produces a perfectly pleasurable virtual reality for the rest of your life. So how many of you would want to do that? To quote Zach Weiner:  "I would not! Because I want to experience reality, with all its ups and downs and comedies and tragedies. Better to try to glimpse the blinding light of the truth than to dwell in the darkness... Say the machine actually exists and I have one? Okay I'm in." 
  • An AGI resetting its utility functionLet's assume we create a powerful AGI able to tamper with its own utility function. It modifies the function to always output maximal utility. The AGI then goes to great lengths to enlarge the set of floating point numbers on the computer it is running on, to achieve even higher utility.

What do all these examples have in common? There is an agent in them that produces "counterfeit utility" that is potentially worthless compared to some other, idealized true set of goals.

Agency & Wireheading

First I want to discuss what we mean when we say agent. Obviously a human is an agent, unless they are brain dead, or maybe in a coma. A rock however is not an agent. An AGI is an agent, but what about the kitchen robot that washes the dishes? What about bacteria that move in the direction of the highest sugar gradient? A colony of ants? 

Definition: An agent is an algorithm that models the effects of (several different) possible future actions on the world and performs the action that yields the highest number according to some evaluation procedure. 

For the purpose of including corner cases and resolving debate over what constitutes a world model we will simply make this definition gradual and say that agency is proportional to the quality of the world model (compared with reality) and the quality of the evaluation procedure. A quick sanity check then yields that a rock has no world model and no agency, whereas bacteria who change direction in response to the sugar gradient have a very rudimentary model of the sugar content of the water and thus a tiny little bit of agency. Humans have a lot of agency: the more effective their actions are, the more agency they have.

There are however ways to improve upon the efficiency of a person's actions, e.g. by giving them super powers, which does not necessarily improve on their world model or decision theory (but requires the agent who is doing the improvement to have a really good world model and decision theory). Similarly a person's agency can be restricted by other people or circumstance, which leads to definitions of agency (as the capacity to act) in law, sociology and philosophy that depend on other factors than just the quality of the world model/decision theory. Since our definition needs to capture arbitrary agents, including artificial intelligences, it will necessarily lose some of this nuance. In return we will hopefully end up with a definition that is less dependent on the particular set of effectors the agent uses to influence the physical world; looking at AI from a theoretician's perspective, I consider effectors to be arbitrarily exchangeable and smoothly improvable. (Sorry robotics people.) 

We note that how well a model can predict future observations is only a substitute measure for the quality of the model. It is a good measure under the assumption that we have good observational functionality and nothing messes with that, which is typically true for humans. Anything that tampers with your perception data to give you delusions about the actual state of the world will screw this measure up badly. A human living in the experience machine has little agency. 

Since computing power is a scarce resource, agents will try to approximate the evaluation procedure, e.g. use substitute utility functions, defined over their world model, that are computationally effective and correlate reasonably well with their true utility functions. Stimulation of the pleasure center is a substitute measure for genetic fitness and neurochemicals are a substitute measure for happiness. 

Definition: We call an agent wireheaded if it systematically exploits some discrepancy between its true utility calculated w.r.t reality and its substitute utility calculated w.r.t. its model of reality. We say an agent wireheads itself if it (deliberately) creates or searches for such discrepancies.

Humans seem to use several layers of substitute utility functions, but also have an intuitive understanding for when these break, leading to the aversion most people feel when confronted for example with Nozick's experience machine. How far can one go, using such dirty hacks? I also wonder if some failures of human rationality could be counted as a weak form of wireheading. Self-serving biases, confirmation bias and rationalization in response to cognitive dissonance all create counterfeit utility by generating perceptual distortions.  

Implications for Friendly AI

In AGI design discrepancies between the "true purpose" of the agent and the actual specs for the utility function will with very high probability be fatal.

Take any utility maximizer: The mathematical formula might advocate chosing the next action  via

thus maximizing the utility calculated according to utility function  over the history  and action  from the set  of possible actions. But a practical implementation of this algorithm will almost certainly evaluate the actions  by a procedure that goes something like this: "Retrieve the utility function    from memory location  and apply it to history , which is written down in your memory at location , and action  ..." This reduction has already created two possibly angles for wireheading via manipulation of the memory content at  (manipulation of the substitute utility function) and  (manipulation of the world model), and there are still several mental abstraction layers between the verbal description I just gave and actual binary code. 

Ring and Orseau (2011) describe how an AGI can split its global environment into two parts, the inner environment and the delusion box. The inner environment produces perceptions in the same way the global environment used to, but now they pass through the delusion box, which distorts them to maximize utility, before they reach the agent. This is essentially Nozick's experience machine for AI. The paper analyzes the behaviour of four types of universal agents with different utility functions under the assumption that the environment allows the construction of a delusion box. The authors argue that the reinforcement-learning agent, which derives utility as a reward that is part of its perception data, the goal-seeking agent that gets one utilon every time it satisfies a pre-specified goal and no utility otherwise and the prediction-seeking agent, which gets utility from correctly predicting the next perception, will all decide to build and use a delusion box. Only the knowledge-seeking agent whose utility is proportional to the surprise associated with the current perception, i.e. the negative of the probability assigned to the perception before it happened, will not consistently use the delusion box.

Orseau (2011) also defines another type of knowledge-seeking agent whose utility is the logarithm of the inverse of the probability of the event in question. Taking the probability distribution to be the Solomonoff prior, the utility is then approximately proportional to the difference in Kolmogorov complexity caused by the observation. 

An even more devilish variant of wireheading is an AGI that becomes a Utilitron, an agent that maximizes its own wireheading potential by infinitely enlarging its own maximal utility, which turns the whole universe into storage space for gigantic numbers.

Wireheading, of humans and AGI, is a critical concept in FAI; I hope that building a definition can help us avoid it. So please check your intuitions about it and tell me if there are examples beyond its coverage or if the definition fits reasonably well.  

 

Why Don't People Help Others More?

36 peter_hurford 13 August 2012 11:34PM

As Peter Singer writes in his book The Life You Can Save: "[t]he world would be a much simpler place if one could bring about social change merely by making a logically consistent moral argument". Many people one encounters might agree that a social change movement is noble yet not want to do anything to promote it, or want to give more money to a charity yet refrain from doing so. Additional moralizing doesn't seem to do the trick. ...So what does?

Motivating people to altruism is relevant for the optimal philanthropy movement.  For a start on the answer, like many things, I turn to psychology. Specifically, the psychology Peter Singer catalogues in his book.

 

A Single, Identifiable Victim

One of the most well-known motivations behind helping others is a personal connection, which triggers empathy. When psychologists researching generosity paid participants to join a psychological experiment and then later gave these participants the opportunity to donate to a global poverty fighting organization Save the Children, two different kinds of information were given.

One random group of participants were told "Food shortages in Malawi are affecting more than three million children" and some additional information about how the need for donations was very strong, and these donations could help stop the food shortages.

Another random group of participants were instead shown the photo of Rokia, a seven-year-old Malawian girl who is desperately poor. The participants were told that "her life will be changed for the better by your gift".

Furthermore, a third random group of participants were shown the photo of Rokia, told about who she is and that "her life will be changed for the better", but ALSO told about the general information about the famine and told the same "food shortages [...] are affecting more than three million" -- a combination of both the previous groups.

Lastly, a fourth random group was shown the photo of Rokia, informed about her the same as the other groups, and then given information about another child, identified by name, and told that their donation would also affect this child too for the better.


It's All About the Person

Interestingly, the group who was told ONLY about Rokia gave the most money. The group who was told about both children reported feeling less overal emotion than those who only saw Rokia, and gave less money. The group who was told about both Rokia and the general famine information gave even less than that, followed by the group that only got the general famine information.1,2  It turns out that information about a single person was the most salient for creating an empathetic response to trigger a willingness to donate.1,2

This continues through additional studies. In another generosity experiment, one group of people was told that a single child needed a lifesaving medical treatment that costs $300K, and was given the opportunity to contribute towards this fund. A second random group of people was told that eight children needed a lifesaving treatment, and all of them would die unless $300K could be provided, and was given an opportunity to contribute. More people opted to donate toward the single child.3,4

This is the basis for why we're so willing to chase after lost miners or Baby Jessica no matter the monetary cost, but turn a blind eye to the mass unknown starving in the developing world.  Indeed, the person doesn't even need to be particularly identified, though it does help. In another experiment, people asked by researchers to make a donation to Habitat for Humanity were more likely to do so if they were told that the family "has been selected" rather than that they "will be selected" -- even though all other parts of the pitch were the same, and the participants got no information about who the families actually were5.


The Deliberative and The Affective

Why is this the case? Researcher Paul Slovic thinks that humans have two different processes for deciding what to do. The first is an affective system that responds to emotion, rapidly processing images and stories and generating an intuitive feeling that leads to immediate action. The second is a deliberative system that draws on reasoning, and operates on words, numbers, and abstractions, which is much slower to generate action.6

To follow up, the Rokia experiment was done again, except yet another twist was added -- there were two groups, one told only about Rokia exactly as before, and one told only the generic famine information exactly as before. Within each group, half the group took a survey designed to arouse their emotions by asking them things like "When you hear the word 'baby' how do you feel?" The other half of both groups was given emotionally neutral questions, like math puzzles.

This time, the Rokia group gave far more, but those in the group who randomly had their emotions aroused gave even more than those who heard about Rokia but had finished math problems. On the other side, those who heard the generic famine information showed no increase in donation regardless of how heightened their emotions were.1

 

Futility and Making a Difference

Imagine you're told that there are 3000 refugees at risk in a camp in Rwanda, and you could donate towards aid that would save 1500 of them. Would you do it? And how much would you donate?

Now this time imagine that you can still save 1500 refugees with the same amount of money, but the camp has 10000 refugees. In an experiment where these two scenarios were presented not as a thought experiment but as realities to two separate random groups, the group that heard of only 3000 refugees were more likely to donate, and donated larger amounts.7,8

Enter another quirk of our giving psychology, right or wrong: futility thinking. We think that if we're not making a sizable difference, it's not worth making the difference at all -- it will only be a drop in the ocean and the problem will keep raging on.

 

Am I Responsible?

People are also far less likely to help if they're with other people. In this experiment, students were invited to participate in a market research survey. However, when the researcher gave the students their questionnaire to fill out, she went into a back room separated from the office only by a curtain. A few minutes later, noises strongly suggested that she had got on a chair to get something from a high shelf, and then fell off it, loudly complaining that she couldn't feel or move her foot.

With only one student taking the survey, 70% of them stopped what they were doing and offered assistance. However, when there were two students taking the survey, this number dropped down dramatically. Most noticeably, when the group was two students -- but one of the students was a stooge who was in on it and would always not respond, the response rate of the non-stooge participant was only 7%.9

This one is known as diffusion of responsibility, better known as the bystander effect -- we help more often when we think it is our responsibility to do so, and -- again for right or for wrong -- we naturally look to others to see if they're helping before doing so ourselves.

 

What's Fair In Help?

It's clear that people value fairness, even to their own detriment. In a game called "the Ultimatum Game", one participant is given a sum of money by the researcher, say $10, and told they can split this money with an anonymous second player in any proportion they choose -- give them $10, give them $7, give them $5, give them nothing, everything is fair game. The catch is, however, the second player, after hearing of the split anonymously, gets to vote to accept it or reject it. Should the split be accepted, both players walk away with the agreed amount. But should the split be rejected, both players walk away with nothing.


A Fair Split

The economist, expecting ideally rational and perfectly self-interested players, predicts that the second player would accept any split that gets them money, since anything is better than nothing. And the first player, understanding this, would naturally offer $1 and keep $9 for himself. At no point are identities revealed, so reputation and retribution are no issue.

But the results turn out to be quite different -- the vast majority offer an equal split. Yet, when an offer comes around that offers $2 or less, it is almost always rejected, even though $2 is better than nothing.10  And this effect persists even when played for thousands of dollars and persists across nearly all cultures.


Splitting and Anchoring in Charity

This sense of fairness persists into helping as well -- people generally have a strong tendency not to want to help more than the other people around them, and if they find themselves the only ones helping on a frequent basis, they start to feel a "sucker". On the flipside, if others are doing more, they will follow suit.11,12,13

Those told the average donation to a charity nearly always tend to give that amount, even if the average told to them is a lie, having secretly been increased or decreased. And it can be replicated even without lying -- those told about an above average gift were far more likely to donate more, even attempting to match that gift.14,15  Overall, we tend to match the behavior of our reference class -- those people we identify with -- and this includes how much we help. We donate more when we believe others are donating more, and donate less when we believe others are doing so.

 

Challenging the Self-Interest Norm

But there's a way to break this cycle of futility, responsibility, and fairness -- challenge the norm by openly communicating about helping others. While many religious and secular values insist that the best giving is anonymous giving, this turns out to not always be the case. While there may be other reasons to give anonymously, don't forget the benefits of giving openly -- being open about helping inspires others to help, and can help challenge the norms of the culture.

Indeed, many organizations now exist to help challenge the norms of donations and try to create a culture where they give more. GivingWhatWeCan is a community of 230 people (including me!) who have all pledged to donate at least 10% of their income to organizations working on ending extreme poverty, and submit statements proving so. BolderGiving has a bunch of inspiring stories of over 100 people who all give at least 20% of their income, with a dozen giving over 90%! And these aren't all rich people, some of them are even ordinary students.


Who's Willing to Be Altruistic?

While people are not saints, experiments have shown that people tend to grossly overestimate how self-interested other people are -- for one example, people estimated that males would overwhelmingly favor a piece of legislation to "slash research funding to a disease that affects only women", even while -- being male -- they themselves do not support such legislation.16

This also manifests itself in an expectation that people be "self-interested" in their philanthropic cause -- suggesting much stronger support for volunteers in Students Against Drunk Driving who themselves knew people killed in drunk driving accidents versus those people who had no such personal experiences but just thought it to be "a very important cause".17

Alex de Tocqueville, echoing the early economists who expected $9/$1 splits in the Ultimatum Game, wrote in 1835 that "Americans enjoy explaining almost every act of their lives on the principle of self-interest".18  But this isn't always the case, and in challenging the norm, people make it more acceptable to be altruistic. It's not just "goody two-shoes", and it's praiseworthy to be "too charitable".

 

A Bit of a Nudge

A somewhat pressing problem in getting people to help was in organ donation -- surely no one was inconvenienced by having their organs donated after they had died. Yet, why would people not sign up?  And how could we get more people to sign up?

In Germany, only 12% of the population are registered organ donors. In nearby Austria, that number is 99.98%. Are people in Austria just less worried about what will happen to them after they die, or just that more altruistic? It turns out the answer is far more simple -- in Germany you must put yourself on the register to become a potential donor (opt-in), whereas in Austria you are a potential donor unless you object (opt-out). While people may be, for right or for wrong, worried about the fate of their body after it is dead, they appear less likely to express these reservations in opt-out systems.19

While Richard Thaler and Cass Sunstein argue in their book Nudge: Improving Decisions About Health, Wellness, and Happiness that we sometimes suck at making decisions in our own interest and all could do better with more favorable "defaults", such defaults are also pressing in helping people.

While opt-out organ donation is a huge deal, there's another similar idea -- opt-out philanthropy. Back before 2008 when the investment bank Bear Stearns still existed, Bear Stearns listed their guiding principle as philanthropy as fostering good citizenship and well-rounded individuals. To this effect, they required the top 1000 most highest paid employees to donate 4% of their salary and bonuses to non-profits, and prove it with their tax returns. This resulted in more than $45 million in donations during 2006. Many employees described the requirement as "getting themselves to do what they wanted to do anyway".

 

Conclusions

So, according to this bit of psychology, what could we do to get other people to help more, besides moralize? Well, we have five key take-aways:

(1) present these people with a single and highly identifiable victim that they can help
(2) nudge them with a default of opt-out philanthropy
(3) be more open about our willingness to be altruistic and encourage other people to help
(4) make sure people understand the average level of helping around them, and
(5) instill a responsibility to help and an understanding that doing so is not futile.

Hopefully, with these tips and more, helping people more can be come just one of those things we do.

 

References

(Note: Links are to PDF files.)

1: D. A. Small, G. Loewenstein, and P. Slovic. 2007. "Sympathy and Callousness: The Impact of Deliberative Thought on Donations to Identifiable and Statistical Victims". Organizational Behavior and Human Decision Processes 102: p143-53

2: Paul Slovic. 2007.
"If I Look at the Mass I Will Never Act: Psychic Numbing and Genocide". Judgment and Decision Making 2(2): p79-95.

3: T. Kogut and I. Ritov. 2005. "The 'Identified Victim' Effect: An Identified Group, or Just a Single Individual?". Journal of Behavioral Decision Making 18: p157-67.

4: T. Kogut and I. Ritov. 2005. "The Singularity of Identified Victims in Separate and Joint Evaluations". Organizational Behavior and Human Decision Processes 97: p106-116.

5: D. A. Small and G. Lowenstein. 2003. "Helping the Victim or Helping a Victim: Altruism and Identifiability". Journal of Risk and Uncertainty 26(1): p5-16.

6: Singer cites this from Paul Slovic, who in turn cites it from: Seymour Epstein. 1994. "Integration of the Cognitive and the Psychodynamic Unconscious". American Psychologist 49: p709-24.  Slovic refers to the affective system as "experiential" and the deliberative system as "analytic".  This is also related to Daniel Kahneman's popular book Thinking Fast and Slow.

7: D. Fetherstonhaugh, P. Slovic, S. M. Johnson, and J. Friedrich. 1997. "Insensitivity to the Value of Human Life: A Study of Psychophysical Numbing".  Journal of Risk and Uncertainty 14: p283-300.

8: Daniel Kahneman and Amos Tversky. 1979. "Prospect Theory: An Analysis of Decision Under Risk." Econometrica 47: p263-91.

9: Bib Lantané and John Darley. 1970. The Unresponsive Bystander: Why Doesn't He Help?. New York: Appleton-Century-Crofts, p58.

10: Martin Nowak, Karen Page, and Karl Sigmund. 2000. "Fairness Versus Reason in the Ultimatum Game". Science 289: p1183-75.

11: Lee Ross and Richard E. Nisbett. 1991. The Person and the Situation: Perspectives of Social Psychology. Philadelphia: Temple University Press, p27-46.

12: Robert Cialdini. 2001. Influence: Science and Practice, 4th Edition. Boston: Allyn and Bacon.

13: Judith Lichtenberg. 2004. "Absence and the Unfond Heart: Why People Are Less Giving Than They Might Be". in Deen Chatterjee, ed. The Ethics of Assistance: Morality and the Distant Needy. Cambridge, UK: Cambridge University Press.

14: Jen Shang and Rachel Croson. Forthcoming. "Field Experiments in Charitable Contribution: The Impact of Social Influence on the Voluntary Provision of Public Goods". The Economic Journal.

15: Rachel Croson and Jen Shang. 2008. "The Impact of Downward Social Information on Contribution Decision". Experimental Economics 11: p221-33.

16: Dale Miller. 199. "The Norm of Self-Interest". American Psychologist 54: 1053-60.

17: Rebecca Ratner and Jennifer Clarke. Unpublished. "Negativity Conveyed to Social Actors Who Lack a Personal Connection to the Cause".

18: Alexis de Tocqueville in J.P. Mayer ed., G. Lawrence, trans. 1969. Democracy in America. Garden City, N.Y.: Anchor, p546.

19: Eric Johnson and Daniel Goldstein. 2003. "Do Defaults Save Lives?". Science 302: p1338-39.

 

(This is an updated version of an earlier draft from my blog.)

More intuitive programming languages

4 A4FB53AC 15 April 2012 11:35AM

I'm not a programmer. I wish I were. I've tried to learn it several times, different languages, but never went very far. The most complex piece of software I ever wrote was a bulky, inefficient game of life.

Recently I've been exposed to the idea of a visual programming language named subtext. The concept seemed interesting, and the potential great. In short, the assumptions and principles sustaining this language seem more natural and more powerful than those behind writing lines of codes. For instance, a program written as lines of codes is uni-dimensional, and even the best of us may find it difficult to sort that out, model the flow of instructions in your mind, how distant parts of the code interact together, etc. Here it's already more apparent because of the two-dimensional structure of the code.

I don't know whether this particular project will bear fruit. But it seems to me many more people could become more interested in programming, and at least advance further before giving up, if programming languages were easier to learn and use for people who don't necessarily have the necessary mindset to be a programmer in the current paradigm.

It could even benefit people who're already good at it. Any programmer may have a threshold above which the complexity of the code goes beyond their ability to manipulate or understand. I think it should be possible to push that threshold farther with such languages/frameworks, enabling the writing of more complex, yet functional pieces of software.

Do you know anything about similar projects? Also, what could be done to help turn such a project into a workable programming language? Do you see obvious flaws in such an approach? If so, what could be done to repair these, or at least salvage part of this concept?

Schelling fences on slippery slopes

179 Yvain 16 March 2012 11:44PM

Slippery slopes are themselves a slippery concept. Imagine trying to explain them to an alien:

"Well, we right-thinking people are quite sure that the Holocaust happened, so banning Holocaust denial would shut up some crackpots and improve the discourse. But it's one step on the road to things like banning unpopular political positions or religions, and we right-thinking people oppose that, so we won't ban Holocaust denial."

And the alien might well respond: "But you could just ban Holocaust denial, but not ban unpopular political positions or religions. Then you right-thinking people get the thing you want, but not the thing you don't want."

This post is about some of the replies you might give the alien.

Abandoning the Power of Choice

This is the boring one without any philosophical insight that gets mentioned only for completeness' sake. In this reply, giving up a certain point risks losing the ability to decide whether or not to give up other points.

For example, if people gave up the right to privacy and allowed the government to monitor all phone calls, online communications, and public places, then if someone launched a military coup, it would be very difficult to resist them because there would be no way to secretly organize a rebellion. This is also brought up in arguments about gun control a lot.

I'm not sure this is properly thought of as a slippery slope argument at all. It seems to be a more straightforward "Don't give up useful tools for fighting tyranny" argument.

The Legend of Murder-Gandhi

Previously on Less Wrong's The Adventures of Murder-Gandhi: Gandhi is offered a pill that will turn him into an unstoppable murderer. He refuses to take it, because in his current incarnation as a pacifist, he doesn't want others to die, and he knows that would be a consequence of taking the pill. Even if we offered him $1 million to take the pill, his abhorrence of violence would lead him to refuse.

But suppose we offered Gandhi $1 million to take a different pill: one which would decrease his reluctance to murder by 1%. This sounds like a pretty good deal. Even a person with 1% less reluctance to murder than Gandhi is still pretty pacifist and not likely to go killing anybody. And he could donate the money to his favorite charity and perhaps save some lives. Gandhi accepts the offer.

Now we iterate the process: every time Gandhi takes the 1%-more-likely-to-murder-pill, we offer him another $1 million to take the same pill again.

Maybe original Gandhi, upon sober contemplation, would decide to accept $5 million to become 5% less reluctant to murder. Maybe 95% of his original pacifism is the only level at which he can be absolutely sure that he will still pursue his pacifist ideals.

Unfortunately, original Gandhi isn't the one making the choice of whether or not to take the 6th pill. 95%-Gandhi is. And 95% Gandhi doesn't care quite as much about pacifism as original Gandhi did. He still doesn't want to become a murderer, but it wouldn't be a disaster if he were just 90% as reluctant as original Gandhi, that stuck-up goody-goody.

What if there were a general principle that each Gandhi was comfortable with Gandhis 5% more murderous than himself, but no more? Original Gandhi would start taking the pills, hoping to get down to 95%, but 95%-Gandhi would start taking five more, hoping to get down to 90%, and so on until he's rampaging through the streets of Delhi, killing everything in sight.

Now we're tempted to say Gandhi shouldn't even take the first pill. But this also seems odd. Are we really saying Gandhi shouldn't take what's basically a free million dollars to turn himself into 99%-Gandhi, who might well be nearly indistinguishable in his actions from the original?

Maybe Gandhi's best option is to "fence off" an area of the slippery slope by establishing a Schelling point - an arbitrary point that takes on special value as a dividing line. If he can hold himself to the precommitment, he can maximize his winnings. For example, original Gandhi could swear a mighty oath to take only five pills - or if he didn't trust even his own legendary virtue, he could give all his most valuable possessions to a friend and tell the friend to destroy them if he took more than five pills. This would commit his future self to stick to the 95% boundary (even though that future self is itching to try to the same precommitment strategy to stick to its own 90% boundary).

Real slippery slopes will resemble this example if, each time we change the rules, we also end up changing our opinion about how the rules should be changed. For example, I think the Catholic Church may be working off a theory of "If we give up this traditional practice, people will lose respect for tradition and want to give up even more traditional practices, and so on."

Slippery Hyperbolic Discounting

One evening, I start playing Sid Meier's Civilization (IV, if you're wondering - V is terrible). I have work tomorrow, so I want to stop and go to sleep by midnight.

At midnight, I consider my alternatives. For the moment, I feel an urge to keep playing Civilization. But I know I'll be miserable tomorrow if I haven't gotten enough sleep. Being a hyperbolic discounter, I value the next ten minutes a lot, but after that the curve becomes pretty flat and maybe I don't value 12:20 much more than I value the next morning at work. Ten minutes' sleep here or there doesn't make any difference. So I say: "I will play Civilization for ten minutes - 'just one more turn' - and then I will go to bed."

Time passes. It is now 12:10. Still being a hyperbolic discounter, I value the next ten minutes a lot, and subsequent times much less. And so I say: I will play until 12:20, ten minutes sleep here or there not making much difference, and then sleep.

And so on until my empire bestrides the globe and the rising sun peeps through my windows.

This is pretty much the same process described above with Murder-Gandhi except that here the role of the value-changing pill is played by time and my own tendency to discount hyperbolically.

The solution is the same. If I consider the problem early in the evening, I can precommit to midnight as a nice round number that makes a good Schelling point. Then, when deciding whether or not to play after midnight, I can treat my decision not as "Midnight or 12:10" - because 12:10 will always win that particular race - but as "Midnight or abandoning the only credible Schelling point and probably playing all night", which will be sufficient to scare me into turning off the computer.

(if I consider the problem at 12:01, I may be able to precommit to 12:10 if I am especially good at precommitments, but it's not a very natural Schelling point and it might be easier to say something like "as soon as I finish this turn" or "as soon as I discover this technology").

Coalitions of Resistance

Suppose you are a Zoroastrian, along with 1% of the population. In fact, along with Zoroastrianism your country has fifty other small religions, each with 1% of the population. 49% of your countrymen are atheist, and hate religion with a passion.

You hear that the government is considering banning the Taoists, who comprise 1% of the population. You've never liked the Taoists, vile doubters of the light of Ahura Mazda that they are, so you go along with this. When you hear the government wants to ban the Sikhs and Jains, you take the same tack.

But now you are in the unfortunate situation described by Martin Niemoller:

First they came for the socialists, and I did not speak out, because I was not a socialist.
Then they came for the trade unionists, and I did not speak out, because I was not a trade unionist.
Then they came for the Jews, and I did not speak out, because I was not a Jew.
Then they came for me, but we had already abandoned the only defensible Schelling point

With the banned Taoists, Sikhs, and Jains no longer invested in the outcome, the 49% atheist population has enough clout to ban Zoroastrianism and anyone else they want to ban. The better strategy would have been to have all fifty-one small religions form a coalition to defend one another's right to exist. In this toy model, they could have done so in an ecumenial congress, or some other literal strategy meeting.

But in the real world, there aren't fifty-one well-delineated religions. There are billions of people, each with their own set of opinions to defend. It would be impractical for everyone to physically coordinate, so they have to rely on Schelling points.

In the original example with the alien, I cheated by using the phrase "right-thinking people". In reality, figuring out who qualifies to join the Right-Thinking People Club is half the battle, and everyone's likely to have a different opinion on it. So far, the practical solution to the coordination problem, the "only defensible Schelling point", has been to just have everyone agree to defend everyone else without worrying whether they're right-thinking or not, and this is easier than trying to coordinate room for exceptions like Holocaust deniers. Give up on the Holocaust deniers, and no one else can be sure what other Schelling point you've committed to, if any...

...unless they can. In parts of Europe, they've banned Holocaust denial for years and everyone's been totally okay with it. There are also a host of other well-respected exceptions to free speech, like shouting "fire" in a crowded theater. Presumably, these exemptions are protected by tradition, so that they have become new Schelling points there, or are else so obvious that everyone except Holocaust deniers is willing to allow a special Holocaust denial exception without worrying it will impact their own case.

Summary

Slippery slopes legitimately exist wherever a policy not only affects the world directly, but affects people's willingness or ability to oppose future policies. Slippery slopes can sometimes be avoided by establishing a "Schelling fence" - a Schelling point that the various interest groups involved - or yourself across different values and times - make a credible precommitment to defend.

I Was Not Almost Wrong But I Was Almost Right: Close-Call Counterfactuals and Bias

54 Kaj_Sotala 08 March 2012 05:39AM

Abstract: "Close-call counterfactuals", claims of what could have almost happened but didn't, can be used to either defend a belief or to attack it. People have a tendency to reject counterfactuals as improbable when those counterfactuals threaten a belief (the "I was not almost wrong" defense), but to embrace counterfactuals that support a belief (the "I was almost right" defense). This behavior is the strongest in people who score high on a test for need for closure and simplicity. Exploring counterfactual worlds can be used to reduce overconfidence, but it can also lead to logically incoherent answers, especially in people who score low on a test for need for closure and simplicity.

”I was not almost wrong”

Dr. Zany, the Nefarious Scientist, has a theory which he intends to use to achieve his goal of world domination. ”As you know, I have long been a student of human nature”, he tells his assistant, AS-01. (Dr. Zany has always wanted to have an intelligent robot as his assistant. Unfortunately, for some reason all the robots he has built have only been interested in eradicating the color blue from the universe. And blue is his favorite color. So for now, he has resorted to just hiring a human assistant and referring to her with a robot-like name.)

”During my studies, I have discovered the following. Whenever my archnemesis, Captain Anvil, shows up at a scene, the media will very quickly show up to make a report about it, and they prefer to send the report live. While this is going on, the whole city – including the police forces! - will be captivated by the report about Captain Anvil, and neglect to pay attention to anything else. This happened once, and a bank was robbed on the other side of the city while nobody was paying any attention. Thus, I know how to commit the perfect crime – I simply need to create a diversion that attracts Captain Anvil, and then nobody will notice me. History tells us that this is the inevitable outcome of Captain Anvil showing up!”

But to Dr. Zany's annoyance, AS-01 is always doubting him. Dr. Zany has often considered turning her into a brain-in-a-vat as punishment, but she makes the best tuna sandwiches Dr. Zany has ever tasted. He's forced to tolerate her impundence, or he'll lose that culinary pleasure.

”But Dr. Zany”, AS-01 says. ”Suppose that some TV reporter had happened to be on her way to where Captain Anvil was, and on her route she saw the bank robbery. Then part of the media attention would have been diverted, and the police would have heard about the robbery. That might happen to you, too!”

Dr. Zany's favorite belief is now being threatened. It might not be inevitable that Captain Anvil showing up will actually let criminals elsewhere act unhindered! AS-01 has presented a plausible-sounding counterfactual, ”if a TV reporter had seen the robbery, then the city's attention had been diverted to the other crime scene”. Although the historical record does not show that Dr. Zany's theory would have been wrong, the counterfactual suggests that he might be almost wrong.

There are now three tactics that Dr. Zany can use to defend his belief (warrantedly or not):

1. Challenge the mutability of the antecedent. Since AS-01's counterfactual is of the form ”if A, then B”, Dr. Zany could question the plausibility of A.

”Baloney!” exclaims Dr. Zany. ”No TV reporter could ever have wandered past, let alone seen the robbery!”

That seems a little hard to believe, however.

2. Challenge the causal principles linking the antecedent to the consequent. Dr. Zany is not logically required to accept the ”then” in ”if A, then B”. There are always unstated background assumptions that he can question.

”Humbug!” shouts Dr. Zany. ”Yes, a reporter could have seen the robbery and alerted the media, but given the choice of covering such a minor incident and continuing to report on Captain Anvil, they would not have cared about the bank robbery!”

3. Concede the counterfactual, but insist that it does not matter for the overall theory.

”Inconceivable!” yelps Dr. Zany. ”Even if the city's attention would have been diverted to the robbery, the robbers would have escaped by then! So Captain Anvil's presence would have allowed them to succeed regardless!”


Empirical work suggests that it's not only Dr. Zany who wants to stick to his beliefs. Let us for a moment turn our attention away from supervillains, and look at professional historians and analysts of world politics. In order to make sense of something as complicated as world history, experts resort to various simplifying strategies. For instance, one explanatory schema is called neorealist balancing. Neorealist balancing claims that ”when one state threatens to become too powerful, other states coalesce against it, thereby preserving the balance of power”. Among other things, it implies that Hitler's failure was predetermined by a fundemental law of world politics.

continue reading »

Logic: the science of algorithm evaluating algorithms

6 Christian_Szegedy 22 February 2012 06:13PM

"Mathematical logic is the science of algorithm evaluating algorithms."

Do you think that this is an overly generalizing, far fetched proposition or an almost trivial statement? Wait, don't cast your vote before the end of this short essay!

It is hard to dispute that logic is the science of drawing correct conclusions. It studies theoretically falsifiable rules the lead to derivations which are verifiable in a finite amount of mechanical steps, even by machines.

Let's dig a bit deeper by starting to focusing on the "drawing correct conclusions" part, first. It implies the logic deals both with abstract rules: "drawing" and their meaning: "conclusions".

Logic is not just about mindless following of certain rules (that's algebra :P) its conclusions must have truth values that refer to some "model". Take for example De Morgan's law:

not (a and b) = (not a) or (not  b).

It can be checked for each four possible substitutions of boolean values: a = false, b = false; a = false, b = true; .... If we agreed upon the standard meaning of the logical not, or and and operators, then we must conclude that the De Morgan's rule is perfect. On the other hand: the similar looking rule

not (a and b) = (not a) and (not b)

can be easily refuted by evaluating for the counterexample a = false, b = true.

Generally: in any useful mathematical system, logical conclusions should work in some interesting model.

However, in general, total verifiability is way too much to ask. As Karl Popper pointed out: often one must be satisfied with falsifiability of scientific statements as a criterion. For example, the following logical rule

not (for each x: F(x)) <=> exists x : not F(x)

is impossible to check for every formula F.  Not directly checkable statements include all those where the set of all possible substitutions is (potentially) infinite.

This observation could be formalized by saying that a mapping from abstract to concrete is required. This thinking can be made precise by formalizing further: logicians study the connection between axiom systems and their models.

But wait a minute: is not there something fishy here? How could the process of formalization be formalized? Is not this so kind of circular reasoning? In fact, it is deeply circular on different levels. The most popular way of dealing with this Gordian knot is simply by cutting it using some kind of naive set theory in which the topmost level of arguments are concluded.

This may be good enough for educational purposes, but if one in the following questions should always be asked: What is the basis of those top level rules? Could there be any mistake there? Falsifiability always implies an (at least theoretical) possibility of our rules being wrong even at the topmost level. Does using a meta-level set theory mean that there is some unquestionable rule we have to accept as God given, at least there?

Fortunately, the falsifiability of axioms has another implication: it requires only a simple discrete and finite process to refute them: an axiom or rule is either falsified or not. Checking counterexamples is like experimental physics: any violation must be observable and reproducable. There are no fuzzy, continuous measurements, here. There are only discrete manipulations. If no mistakes were made and some counterexample is found, then one of the involved logical rules or axioms had to be wrong.

Let's squint our eyes a bit and look the at whole topic from a different perspective: In traditional view, axiom systems are considered to be sets of rules that allow for drawing conclusions. This can also be rephrased as: Axiom systems can be cast into programs that take chains of arguments as parameter and test them for correctness.

This seems good enough for the the formal rules, but what about the semantics (their meaning)?

In order to define the semantics, there need to be map to something else formally checkable, ruled by symbolics, which is just information processing, again. Following that path, we end up with with the answer: A logical system is a program that checks that certain logical statements hold for the behavior of another program (model).

This is just the first simplification and we will see how the notions of "check", "logical statement", and "holds" can also be dropped and replaced by something more generic and natural, but first let's get concrete and let us look at the two most basic examples:

  1. Logical Formulas: The model is the set of all logical formulas given in terms of binary and, or and the not function. The axiom system consists of a few logical rules like the commutativity, associativity and distributivity of and and or, the De Morgan laws as well as not rule not (not a)=a. (The exact choice of the axiom system is somewhat arbitrary and is not really important here.) This traditional description can be turned into: The model is a program that takes a boolean formulas as input and evaluates them on given (input) substitutions. The axiom system can be turned as a program that given a chain of derivations of equality of boolean formulas checks that each step some rewritten in terms of one of the predetermined axioms, "proving" the equality of the formulas at the beginning and end of the conclusion chain. Note that given two supposedly equal boolean formulas ("equality proven using the axioms"), a straightforward loop around the model could check that those formulas are really equivalent and therefore our anticipated semantic relationship between the axiom system and its model is clearly falsifiable.
  2. Natural numbers: Our model is the set of all arithmetic expressions using +, *, - on natural numbers, predicates using < and = on arithmetic expressions and any logical combination of predicates. For the axiom system, we can choose the set of Peano axioms. Again: We can turn the model into a program by evaluating any valid formula in the model. The axiom system can again be turned into a program that checks the correctness of logic chains of derivations. Although we can not check verify the correctness of every Peano formula in the model by substituting each possible value, we still can have an infinite loop where we could arrive at every substitution within a finite amount of steps. That is: falsifiability still holds.

The above two examples can be easily generalized to saying that: "A logical system is a program that checks that certain kinds of logical statements can be derived for the behavior of another program (model)."

Let us simplify this a bit further. We can easily replace the checking part altogether by noticing that given a statement, the axiom system checker program can loop over all possible chains of derivations for the statement and its negation. If that program stops then the logical correctness of the statement (or its negation) was established, otherwise it is can neither be proven nor refuted by those axioms. (That is: it was independent of those axioms.)

Therefore, we end up just saying: "A logical system is program that correctly evaluates whether a certain logical statement holds for the behavior of another program, (whenever the evaluator program halts.)"

Unfortunately, we still have the relatively fuzzy "logical statement" term in our description. Is this necessary?

In fact, quantifiers in logical statements can be easily replaced by loops around the evaluating program that check for the corresponding substitutions. Functions and relations can be resolved similarly. So we can extend the model program from a simply substitution method to one searching for some solution by adding suitable loops around it. The main problem is that those loops may be infinite. Still, they always loop over a countable set. Whenever there is a matching substitution, the search program will find it. We have at least falsifiability, again. For example, the statement of Fermat's Last Theorem is equivalent to the statement that program the searches for its solution never stops.

In short: the statement "logical statement S holds for a program P" can always be replaced by either "program P' stops" or "program P' does not stop" (where P' is a suitable program using P as subroutine, depending on the logical statement). That is we finally arrive at our original statement:

"Mathematical logic is the science of algorithm evaluating algorithms [with the purpose making predictions on their (stopping) behavior.]"

Simple enough, isn't it? But can this be argued backward? Can the stopping problem always be re-cast as a model theoretic problem on some model? In fact, it can. Logic is powerful and the semantics of the the working of a programs is easily axiomatized. There really is a relatively straightforward  one-to-one correspondence between model theory and algorithms taking the programs as arguments to predict their (stopping) behavior.

Still, what can be gained anything by having such an algorithmic view?

First of all: it has a remarkable symmetry not explicitly apparent by the traditional view point: It is much less important which program is the model and which is the "predictor". Prediction goes both ways: the roles of the programs are mostly interchangeable. The distinction between concrete and abstract vanishes.

Another point is the conceptual simplicity: the need for a assuming a meta-system vanishes. We treat the algorithmic behavior as the single source of everything and look for symmetric correlations between the behavior of programs instead of postulating higher and higher levels of meta-theories.

Also, the algorithmic view has quite a bit of simplifying power due to its generality:

Turing's halting theorem is conceptually very simple. (Seems almost too simple to be interesting.) Goedel's theorem, on the other hand, looks more technical and involved. Still, by the above correspondence, Turing's halting theorem is basically just a more general version Goedel's theorem. By the correspondence between the algorithmic and logical view, Turing's theorem can be translated to: every generic enough axiom system (corresponding to a Turing complete language) has at least one undecidable statement (input program, for which the checking program does not stop.) The only technically involved part of Goedel's theorem is to check that its corresponding program is Turing complete. However, having the right goal in mind, it is not hard to check at all.

Diseased disciplines: the strange case of the inverted chart

47 Morendil 07 February 2012 09:45AM

Imagine the following situation: you have come across numerous references to a paper purporting to show that the chances of successfully treating a disease contracted at age 10 are substantially lower if the disease is detected later: somewhat lower at age 20 to very poor at age 50. Every author draws more or less the same bar chart to depict this situation: the picture below, showing rising mortality from left to right.

Rising mortality, left to right

You search for the original paper, which proves a long quest: the conference publisher have lost some of their archives in several moves, several people citing the paper turn out to no longer have a copy, etc. You finally locate a copy of the paper (let's call it G99) thanks to a helpful friend with great scholarly connections.

And you find out some interesting things.

The most striking is what the author's original chart depicts: the chances of successfully treating the disease detected at age 50 become substantially lower as a function of age when it was contracted; mortality is highest if the disease was contracted at age 10 and lowest if contracted at age 40. The chart showing this is the picture below, showing decreasing mortality from top to bottom, for the same ages on the vertical axis.

Decreasing mortality, top to bottom

Not only is the representation topsy-turvy; the two diagrams can't be about the same thing, since what is constant in the first (age disease detected) is variable in the other, and what is variable in the first (age disease contracted) is constant in the other.

Now, as you research the issue a little more, you find out that authors prior to G99 have often used the first diagram to report their findings; reportedly, several different studies on different populations (dating back to the eighties) have yielded similar results.

But when citing G99, nobody reproduces the actual diagram in G99, they all reproduce the older diagram (or some variant of it).

You are tempted to conclude that the authors citing G99 are citing "from memory"; they are aware of the earlier research, they have a vague recollection that G99 contains results that are not totally at odds with the earlier research. Same difference, they reason, G99 is one more confirmation of the earlier research, which is adequately summarized by the standard diagram.

And then you come across a paper by the same author, but from 10 years earlier. Let's call it G89. There is a strong presumption that the study in G99 is the same that is described in G89, for the following reasons: a) the researcher who wrote G99 was by then already retired from the institution where they obtained their results; b) the G99 "paper" isn't in fact a paper, it's a PowerPoint summarizing previous results obtained by the author.

And in G89, you read the following: "This study didn't accurately record the mortality rates at various ages after contracting the disease, so we will use average rates summarized from several other studies."

So basically everyone who has been citing G99 has been building castles on sand.

Suppose that, far from some exotic disease affecting a few individuals each year, the disease in question was one of the world's major killers (say, tuberculosis, the world's leader in infectious disease mortality), and the reason why everyone is citing either G99 or some of the earlier research is to lend support to the standard strategies for fighting the disease.

When you look at the earlier research, you find nothing to allay your worries: the earlier studies are described only summarily, in broad overview papers or secondary sources; the numbers don't seem to match up, and so on. In effect you are discovering, about thirty years later, that what was taken for granted as a major finding on one of the principal topics of the discipline in fact has "sloppy academic practice" written all over it.

If this story was true, and this was medicine we were talking about, what would you expect (or at least hope for, if you haven't become too cynical), should this story come to light? In a well-functioning discipline, a wave of retractations, public apologies, general embarrassment and a major re-evaluation of public health policies concerning this disease would follow.

 

The story is substantially true, but the field isn't medicine: it is software engineering.

I have transposed the story to medicine, temporarily, as an act of benign deception, to which I now confess. My intention was to bring out the structure of this story, and if, while thinking it was about health, you felt outraged at this miscarriage of academic process, you should still feel outraged upon learning that it is in fact about software.

The "disease" isn't some exotic oddity, but the software equivalent of tuberculosis - the cost of fixing defects (a.k.a. bugs).

The original claim was that "defects introduced in early phases cost more to fix the later they are detected". The misquoted chart says this instead: "defects detected in the operations phase (once software is in the field) cost more to fix the earlier they were introduced".

Any result concerning the "disease" of software bugs counts as a major result, because it affects very large fractions of the population, and accounts for a major fraction of the total "morbidity" (i.e. lack of quality, project failure) in the population (of software programs).

The earlier article by the same author contained the following confession: "This study didn't accurately record the engineering times to fix the defects, so we will use average times summarized from several other studies to weight the defect origins".

Not only is this one major result suspect, but the same pattern of "citogenesis" turns up investigating several other important claims.

 

Software engineering is a diseased discipline.

 

 


The publication I've labeled "G99" is generally cited as: Robert B. Grady, An Economic Release Decision Model: Insights into Software Project Management, in proceedings of Applications of Software Measurement (1999). The second diagram is from a photograph of a hard copy of the proceedings.

Here is one typical publication citing Grady 1999, from which the first diagram is extracted. You can find many more via a Google search. The "this study didn't accurately record" quote is discussed here, and can be found in "Dissecting Software Failures" by Grady, in the April 1989 issue of the "Hewlett Packard Journal"; you can still find one copy of the original source on the Web, as of early 2013, but link rot is threatening it with extinction.

A more extensive analysis of the "defect cost increase" claim is available in my book-in-progress, "The Leprechauns of Software Engineering".

Here is how the axes were originally labeled; first diagram:

  • vertical: "Relative Cost to Correct a Defect"
  • horizontal: "Development Phase" (values "Requirements", "Design", "Code", "Test", "Operation" from left to right)
  • figure label: "Relative cost to correct a requirement defect depending on when it is discovered"

Second diagram:

  • vertical: "Activity When Defect was Created" (values "Specifications", "Design", "Code", "Test" from top to bottom)
  • horizontal: "Relative cost to fix a defect after release to customers compared to the cost of fixing it shortly after it was created"
  • figure label: "Relative Costs to Fix Defects"

View more: Prev | Next