Robustness of Cost-Effectiveness Estimates and Philanthropy
Note: I formerly worked as a research analyst at GiveWell. This post describes the evolution of my thinking about robustness of cost-effectiveness estimates in philanthropy. All views expressed here are my own.
Up until 2012, I believed that detailed explicit cost-effectiveness estimates are very important in the context of philanthropy. My position was reflected in a comment that I made in 2011:
The problem with using unquantified heuristics and intuitions is that the “true” expected values of philanthropic efforts plausibly differ by many orders of magnitude, and unquantified heuristics and intuitions are frequently insensitive to this. The last order of magnitude is the only one that matters; all others are negligible by comparison. So if at all possible, one should do one’s best to pin down the philanthropic efforts with the “true” expected value per dollar of the highest (positive) order of magnitude. It seems to me as though any feasible strategy for attacking this problem involves explicit computation.
During my time at GiveWell, my position on this matter shifted. I still believe that there are instances in which rough cost-effectiveness estimates can be useful for determining good philanthropic foci. But I’ve shifted toward the position that effective altruists should spend much more time on qualitative analysis than on quantitative analysis in determining how they can maximize their positive social impact.
In this post I’ll focus on one reason for my shift: explicit cost-effectiveness estimates are generally much less robust than I had previously thought.
Being Foreign and Being Sane
I've been reading Less Wrong for a while now, and have recently been casting about for suitable topics to write on. I've decided to break the ice now with an essay on what living and working abroad in Korea has taught me which carries over into studying rationality. While more personal than technical, this inaugural post contains generalizable lessons that I think will be of interest to anyone trying to improve their thinking.
You may be skeptical, so let me briefly make my case that traveling offers something to the aspiring rationalist. Many have written about the benefits of traveling, but for our purposes here is what matters:
Being abroad can make certain important concepts in rationality a part of you in ways studying can't match.
It's easy to read -- and to really believe -- that the map is not the territory, say, without it changing how you actually act. Information often gathers dust on the shelves in your frontal lobe without ever making it into the largely unconscious bits of your brain where so much of your deciding takes place.
With this in mind travel can be seen as part of the class of efforts to learn rationality without directly studying the science, instead doing something like playing Go or poker, for example. I don't know for sure, but such efforts could hold the promise of teaching us to incorporate insights into emotional attachment, statistical probabilities, strategy, maximizing utility, and the like -- things we've known for a long time -- into our instincts, deep down where they can actually change how we behave.
I say all this because what living in a foreign country has given me is not so much a software update which has remade me into a paragon of rationality, but rather a hearty appreciation for certain facts which might make my thought-improvement efforts more fruitful. No doubt many of you have already long-ago internalized all of this, and for you I won't be saying anything very profound.
Nevertheless, here is what I've learned:
1) You are vastly more complicated than you think you are.
The proposal for the Dartmouth conference of 1956, considered by some to be the birth of the field of AI research, had this to say:
An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves. We think that a significant advance can be made in one or more of these problems if a carefully selected group of scientists work on it together for a summer.
Not to deny that considerable progress has been made in the past half century, but I think we can all agree that this thinking was just a tad bit optimistic.
I'm not an expert on AI research history, but it seems reasonable to assume that these proto-AI researchers perhaps didn't appreciate how complex humans are. You look at a triangle and you see a triangle; you reach for a coffee cup and grasp it; you start speaking a sentence and finish it with only the occasional pause. What could be simpler? We all forget our car keys sometimes, and some of us know a little bit about bizarre neurological problems like aphasia, but still. In general we function so well that it never occurs to us that the things we do might actually be difficult to implement.
The problem runs deeper than this, though, because there doesn't seem to be much in the way of techniques for elucidating this complexity from the inside. If there were, neuroscience might've been discovered a millennium ago in East Asia by Buddhist adepts. But instead our efforts at aiming the introspective flashlights on the machinery of our minds are thwarted by their presence totally outside our conscious awareness.
Well, if you ever feel like you're not fully appreciating the intricacies of your wetware, sit in a coffee shop or bus stop in a foreign country while eavesdropping on people whose effortless bantering could not be more inscrutable, and you'll have it impressed upon you. Alternatively, try to explain to someone with little-to-no English knowledge what something like "simple" or "almost all of" means. Even without a bit of neuroscience training you'll start to get a grasp on the vastness of the gears and levers that make every utterance possible.
This insight, at least for me, seems to creep into the rest of your thinking life, though in my case it's hard to tell because I've always pondered things like this. It isn't a far leap from here to see the potential value of research into topics like Friendly AI. If human language and vision are complicated, what are the chances that human value systems are simple? If you didn't manage to notice your retinal blind spot or the mechanisms by which you conjugate verbs in your native tongue, what are the chances that you aren't at least a little mistaken about your true goals and desires and how best to achieve them? Exactly. So maybe it's time to start reading those sequences, eh?
2) Don't be bewitched by words
Obviously if you go to a country where English or a different language you're already fluent in is spoken, this won't apply as much. But my experience has shown me that living in and learning a foreign language bestows several valuable insights on those intrepid enough to stick with it. Simply put, a sufficiently reflective and intelligent person could independently figure out about half of the sequence A Human's Guide to Words just by being in a foreign country and thinking about the experience.
First you'd have to go through the shocking revelation that so much of what you say is a fairly arbitrary set of language conventions, and then you'd begin to relearn how to communicate. You'd come to realize that words are mental paintbrush handles with which you guide the attention of other humans to certain clusters in thingspace, and that they are often disgusied queries with hidden connotations. This will be triply reinforced by the fact that you'd often have to resort to empiricism to get your point across - accompanying the word 'red' or 'chair' by actually point to red things or chairs. If you're spending time with natives the inverse will happen, and they will have to point to the parts of the world that words represent to communicate. You'll have a head start in replacing the symbol with the substance because you'll be playing taboo with nearly every word you know. Since you'll be doing this with low-level language, it'll require elbow grease to port this into your native tongue when discussing topics like free will. But if you can avoid slipping into cached thoughts, the training you received when you were a foreigner will likely prove useful.
Beyond this, however, is the tantalizing possibility that we may be more rational when we think in a foreign language, perhaps because it increases reliance on the slow, analytic System 2 at the expense of the rapid-fire, emotional System 1. Psychologists from the University of Chicago tested this idea using English speakers proficient in Japanese, Korean speakers proficient in English, and English speakers proficient in French (Keysar, hayakawa, & An, 2011) [NOTE: I'm aware this study has been mentioned before on Less Wrong, but I believe this is the first actual discussion of the experiment and its methodology]. In the first few experiments participants were randomly sorted into two groups, one of which was given a test in their native language and one of which was given a test in the foreign language. These tests were designed to elicit a well-known tendency for humans to differ in their risk preference depending on how the situation is framed.
Here's how it works: imagine that you turn on the news today to find out that an exotic new disease is ravaging Asia, with an expected final death toll of 600,000. The governments of the world decided that the best solution would be to design two separate drugs, and then to randomly select one reader of Less Wrong to decide between the two. Your number came up, and now you have a choice to make.
Drug A is guaranteed to save 200,000 people. Drug B has a 33% chance of saving everyone and a 66% chance of saving no one.
This is called the gain-framing, because what's emphasized is how many lives you'll save, or gain. When framed this way, people often prefer to administer Drug A. But studies find that if the same problem is loss-framed - that is, with drug A it is guaranteed that 400,000 people die while with Drug B there is a 33% chance that no one will die and a 66% that everyone will - far fewer people prefer Drug A, even though the results of using the drugs are identical.
Besides being sorted by foreign language participants were also randomly sorted by whether or not they got the gain or loss framing. Participants tested in their native language showed the predicted bias, but when tested in the foreign language, about an equal number of people preferred Drug A and Drug B.
An additional study found the same effect of foreign language on reasoning, but using a different bias. People tend to be loss averse, preferring to avoid a loss more than they prefer to gain an identical (or slightly better) amount. This means that people will often turn down an even bet which holds the possibility of gaining $12 and the possibility of losing $10, even though this bet has positive expected value. As with the other studies, Korean speakers proficient in English more often showed this tendency when reasoning in their native language than when reasoning in a foreign one, especially for larger bets.
There are a million reasons to learn a foreign language, but it'd be a very costly way to improve rationality. With that said, for anyone willing to invest the time and effort, better thinking could be the outcome. But even if you don't go to the trouble, simply trying to communicate with people who don't speak the same language as you will teach you a lot about how cognition and communication work.
3) The Zen of the Unfamiliar
Living in another culture can make you aware of so many things that you previously failed to notice at all. I remember not long after I got to Korea, I was in my kitchen and noticed that my sink was different from any of the ones I'd seen back in the States. It was a single open pit sunk into the counter, with a strange spinning mechanism where the drain usually is. After investigating for a while, I realized two things: one, the spinning mechanism was actually a multi-part contraption meant to catch food before it went down the drain (no idea why it could spin) and two, I'd just spend 100 times longer thinking about sinks than I had in the rest of my life combined.
To successfully live in a foreign country you'll have to master the art of noticing things fairly quickly. You'll start to watch how people dress, how they talk, how close they stand to each other, the relative frequency of eye contact, how they chew their food, what order people get served drinks. You'll learn to read the environment to learn where to stand in line, where to catch the bus, where and how to buy things, which door is the exit and which one the entrance, whether or not certain places are likely to be safe, etc.
You'll accomplish most of this by gathering evidence, forming hypotheses, using induction and deduction, and updating on new evidence. The things you've been reading about on Less Wrong will be put to use in finding food and shelter, the tools of rationality will be your compass in a world where you can't read what's written on signs or buildings and most people can't understand your questions. So there's a box on your wall with three buttons, two dials, a bunch of lights, and you're pretty sure it can make hot water come out of the shower? Not a word of English anywhere on it, you say? Well then you'll have to change one variable at a time and take note of the results, like any good scientist would.
Being immersed in a set of shared cultural and linguistic norms that you don't understand makes almost every aspect of your life an experiment. It's exhausting, and one of the most informative experiences I've ever had. On an emotional level, it will teach you to be more at ease with partial understanding, frustration, and confusion. With your comfort zone an ocean away, you'll either persevere and think on your feet, or you'll end up sleeping in the rain.
__
Like with learning a foreign language, there are many reasons to travel abroad and experience another culture. And of course, a plane ticket alone is not enough to make you a better thinker. But if you know what to look for and are actively seeking to grow from the experience, I can attest that being foreign for a little while is one way to become a bit more sane.
Mathematicians and the Prevention of Recessions
Note: I completed a PhD in Mathematics from University of Illinois under the direction of Nathan Dunfield in 2011. I worked as a research analyst at GiveWell from April 2012 to May 2013. All views expressed here are my own.
About this post: I've long been interested in ways in which mathematicians can contribute high social value. In this post, I discuss a tentative idea along these lines. My thoughts are very preliminary in nature, and my intent in making this post is to provide a launching point for further exploration of the subject, rather than to persuade.
Recessions as a serious threat to global welfare
In 2008, the US housing bubble popped, precipitating the Great Recession. The costs of this were staggering:
- It’s been claimed that the cost to US taxpayers in bank bailouts was $9 trillion.
- The Dow Jones Industrial Average dropped by almost 50% and took over 4 years to recover.
- US unemployment jumped from ~5% to ~10%, and has only gradually been declining.
- Budget cuts were especially great for government support of activities with unusually high humanitarian value to those without political constituency, such as investment in global health.
-
It’s been claimed that recessions cause a drop in prosocial behavior.
All told, the Great Recession had massive negative humanitarian disvalue, and preventing another such recession would have massive humanitarian value.
Transparent financial analysis as a possible solution
There are actors in finance who accurately predicted that there was a housing bubble that was on the brink of popping, and who bet heavily against subprime mortgages, reaping enormous profits as a result. The most prominent example is John Paulson, who made $3.7 billion in a 2007 alone, starting from a base of less than $1 billion. There are less extreme examples that are nevertheless very striking.
It’s difficult to determine the relative roles that skill and luck played in these peoples’ success, and the situation is further obscured by hindsight bias. Nevertheless, it seems possible that the financial success of Paulson and others was a consequence of careful analysis and shrewdness, and that other people of sufficiently high intellectual caliber and rationality would have been able to predict it as well.
As is always the case in finance, those who recognized the impending pop of the housing bubble kept their analysis secret, because sharing it would have allowed others to partially close the arbitrage opportunity, reducing the potential to profit. If these people had made their thinking public, it could have resulted in other people betting against the housing bubble earlier on, popped the housing bubble when it was smaller, possibly substantially lessening the severity of the ensuing recession. While there were people who publicly voiced concern, a large number of people would have had a bigger impact
This suggests that transparent financial analysis by intellectual elites could carry massive humanitarian value.
Mathematicians as unusually well positioned to perform such analysis
In the course of my graduate school days, I became familiar with mathematical community. There’s a wide cultural gulf between pure math and finance. My experience was that mathematicians generally view finance as “dirty business,” on account of:
- Often having left-wing political beliefs
- Discomfort with the zero-sum and/or negative-sum nature of finance
- Not identifying with materialism
- Disliking messy problems that are less intrinsically interesting than problems in pure math.
I believe that this gulf has led to a potential opportunity being overlooked: mathematicians may be ideally suited to perform transparent financial analysis that reduces damage from financial bubbles.
This idea occurred to me a few weeks ago. Ideas for philanthropic interventions generally fall apart upon closer examination, and so I wasn’t too optimistic about it holding up. So I was surprised when Neal Koblitz (co-creator of elliptic curve cryptography) raised the same idea in unrelated correspondence:
If mathematicians had been noticing the dubious ways that people in the financial world were claiming to be applying mathematics, and if they had publicly and loudly criticized the misuse of mathematics, then the world might have been spared the collapse of 2008 (or, rather, it wouldn't have been as bad). If mathematicians could have played a role stopping the credit-derivatives bubble before it got out of hand, the economic value of doing that would have been in the trillions of dollars.
When an idea occurs to two people independently, the case for it being a good idea is strengthened. Moreover, Koblitz has a long history of involvement with humanitarian efforts and so can be expected to have perspective on them.
Some reasons why mathematicians seem unusually well suited to the task are:
Transferable Skills — Most mathematicians are unfamiliar with some of most important tools used in finance: statistics, data analysis & programming. But there’s a historical track record of mathematicians being able to pick up these skills and use them to powerful effect. James Simons transitioned from differential geometry to quantitative finance, and became one of the most successful hedge fund managers ever. Cathy O’Neil did a PhD in algebraic number theory under Barry Mazur’s direction, and got a job at DE Shaw, which is one of the most prestigious hedge funds. Mathematicians who are motivated to learn these skills are well positioned to do so.
There are other skills that are very important for successful financial analysis – in particular, one has to have a good eye for empirical data. This is a skill that’s not directly transferable, but it still seems likely that a nontrivial fraction of mathematicians could develop high facility with it.
Intellectual Caliber — The mathematics community has a very dense concentration of intellectual power. James Simons offers a direct point of comparison between math and finance:
Simons won the Oswald Veblan Prize in Geometry before leaving academia to start Renaissance Technologies. There are 25 living mathematicians who have won this prize. The prize is awarded exclusively for work in geometry/topology, and if one looks more broadly at all mathematical fields, one can generate a list of about 100 living mathematicians who were at least as accomplished as Simons at the same age.
After leaving academia, Simons made $10 billion in quantitative finance. What I find most interesting about this is that the situation is not that Simons succeeded where other mathematicians of the same caliber had failed – rather, Simons is virtually the only pure mathematician of his caliber to have left academia. This raises the possibility that there are a handful of elite mathematicians who could make much better financial predictions than most present day actors in finance. Less accomplished but capable mathematicians may also do very well.
Cautiousness — Mathematicians are naturally intellectually conservative, as they spend much of their time rigorously examining arguments for flaws. Thus, they’re unusually unlikely to succumb to greed and fear, which are factors that are thought to play a large role in the behavior of financial markets, and which lead to speculative bubbles. This is corroborated by some of Cathy O’Neil’s remarks on finance.
Implications
The above considerations suggest that mathematicians could contribute enormous social value by engaging in transparent financial analysis.
Many mathematicians who I know wish that they could contribute more social value. In the essay Is there beauty in mathematical theories?, the great mathematician Robert Langlands wrote:
In a letter to A.-M.Legendre of 1830, which I came across while preparing this lecture, Jacobi famously wrote
It is true that Mr. Fourier thought that the principal goal of mathematics was their public utility and their use in explaining natural phenomena. A philosopher like him should have known that the only goal of Science is the honor of the human spirit, and that as such, a question in number theory is worth a question concerning the system of the world.
I am not sure it is so easy. I have given a great deal of my life to matters closely related to the theory of numbers, but the honor of the human spirit is, perhaps, too doubtful and too suspect a notion to serve as vindication. […] Moreover, the appeal to the common welfare as a goal of mathematics is, if not then at least now, often abusive. So it is not easy to find an apology for a life in mathematics.
A fair number of mathematicians don’t have any choice but to do pure math. Gromov wrote:
You become a mathematician, a slave of this insatiable hunger of your brain, of everybody's brain, for making structures of everything that goes into it.
I'm very sympathetic to Gromov's remark, and I think that for people who constituted in this way, it’s probably best not to try to suppress these urges, as such attempts tend to be unsustainable and result in lower contributions to global welfare rather than higher ones.
But for mathematicians who are:
- Tenured professors who don’t have to worry about career considerations
- Able to enjoy financial analysis
- Strongly motivated to do an excellent job
there may be a major opportunity to contribute enormous social value by conducting transparent high quality financial analysis.
This question warrants further investigation.
Could Robots Take All Our Jobs?: A Philosophical Perspective
Note: The following is a draft of a paper written with an audience of philosophers in mind. It focuses on answering objections to AI likely to be made by contemporary philosophers, but it is still likely to be of interest to readers of LessWrong for obvious reasons, and I've tried to avoid assuming any specialized philosophical background.
The title of this paper probably sounds a little strange. Philosophy is generally thought of as an armchair discipline, and the question of whether robots could take all our jobs doesn’t seem like a question that can be settled from the armchair. But it turns out that when you look at this question, it leads you to some other questions that philosophers have had quite a bit to say about.
Some of these other questions are conceptual. They seem like they could in fact be answered from the armchair. Others are empirical but very general. They seem like they require going out and looking at the world, but they’re not about specific technologies. They’re about how the universe works, how the mind works, or how computers work in general. It’s been suggested that one of the distinctive things about philosophical questions is their level of generality. I don’t know whether that’s true, but I do know that some of these general empirical questions are ones that philosophers have had quite a bit to say about.
The line of reasoning in this paper is similar to arguments discussed by Alan Turing (1950), Hubert Dreyfus (1992), and David Chalmers (2010). I won’t say much about those discussions, though, for reasons of space and also because I’ll frame the discussion a bit differently. I want to avoid debates about what “intelligence” is, what “information processing” is, or what it would mean to say the brain is a machine. Hence the focus on machines taking human jobs. I should mention that I’m not the first to suggest this focus; after writing the first draft of this paper I found out that one AI researcher had already proposed replacing the “Turing test” with an “employment test” (Nilsson 2005).
Here, I’m going to assume no job has “done by a human” as part of its definition. I realize that in the future, there may be demand for having jobs specifically done by humans. People might want to be served by human bartenders even if robot bartenders do just as good of a job, in the same way that some people prefer handmade goods even when mass-produced ones are cheaper for the same or better quality (Doctorow 2011). But having acknowledged this issue, I’m going to spend the rest of the paper ignoring it.
Orwell and fictional evidence for dictatorship stability
"If you want a picture of the future, imagine a boot stamping on a human face—forever."
George Orwell (Eric Arthur Blair), Nineteen Eighty-Four
Orwell's Nineteen Eighty-Four is brilliant, terrifying and useful. It's been at its best fighting against governmental intrusions, and is often quoted by journalists and even judges. It's cultural impact has been immense. And, hey, it's well written.
But that doesn't mean it's accurate as a source of predictions or counterfactuals. Orwell's belief that "British democracy as it existed before 1939 would not survive the war" was wrong. Nineteen Eighty-Four did not predict the future course of communism. There is no evidence that anything like the world he envisaged could (or will) happen. Which isn't the same as saying that it couldn't, but we do require some evidence before accepting Orwell's world as realistic.
Yet from this book, a lot of implicit assumptions have seeped into our consciousness. The most important one (shared with many other dystopian novels) is that dictatorships are stable forms of government. Note the "forever" in the quote above - the society Orwell warned about would never change, never improve, never transform. In several conversations (about future governments, for instance), I've heard - and made - the argument that a dictatorship was inevitable, because it's an absorbing state. Democracies can come become dictatorships, but dictatorships (barring revolutions) will endure for good. And so the idea is that if revolutions become impossible (because of ubiquitous surveillance, for instance), then we're stuck with Big Brother for life, and for our children's children'c children's lives.
But thinking about this in the context of history, this doesn't seem credible. The most stable forms of government are democracies and monarchies; nothing else endures that long. And laying revolutions aside, there have been plenty of examples of even quite nasty governments improving themselves. Robespierre was deposed from within his own government - and so the Terror, for all its bloodshed, didn't even last a full year. The worse excesses of Stalinism ended with Stalin. Gorbachev voluntarily opened up his regime (to a certain extent). Mao would excoriate the China of today. Britain's leaders in the 19th and 20th century gradually opened up the franchise, without ever coming close to being deposed by force of arms. The dictatorships of Latin America have mostly fallen to democracies (though revolutions played a larger role there). Looking over the course of recent history, I see very little evidence the dictatorships have much lasting power at all - or that they are incapable of drastic internal change and even improvements.
Now, caveats abound. The future won't be like the past - maybe an Orwellian dictatorship will become possible with advanced surveillance technologies. Maybe a world government won't see any neighbouring government doing a better job, and feel compelled to match it by improving lot of its citizens. Maybe the threat of revolution remains necessary, even if revolts don't actually happen.
Still, we should refrain from assuming that dictatorships, whether party or individual, are somehow the default state, and conduct a much more evidence-based analysis of the matter.
Is a paperclipper better than nothing?
Thought experiment:
Through whatever accident of history underlies these philosophical dilemmas, you are faced with a choice between two, and only two, mutually exclusive options:
* Choose A, and all life and sapience in the solar system (and presumably the universe), save for a sapient paperclipping AI, dies.
* Choose B, and all life and sapience in the solar system, including the paperclipping AI, dies.
Phrased another way: does the existence of any intelligence at all, even a paperclipper, have even the smallest amount of utility above no intelligence at all?
If anyone responds positively, subsequent questions would be which would be preferred, a paperclipper or a single bacteria; a paperclipper or a self-sustaining population of trilobites and their supporting ecology; a paperclipper or a self-sustaining population of australopithecines; and so forth, until the equivalent value is determined.
New LW Meetups: Bristol, Tel Aviv
This summary was posted to LW main on May 17th. The following week's summary is here.
New meetups (or meetups with a hiatus of more than a year) are happening in:
- First Bristol meetup: 25 May 2013 03:00PM
- Tel Aviv, Israel Meetup - Goal Clarification with special guest Cat from CFAR: 23 May 2013 07:00PM
Other irregularly scheduled Less Wrong meetups are taking place in:
- Atlanta Lesswrong's May Meetup: The Rationality of Social Relationships, Friendship, Love, and Family.: 17 May 2013 07:00PM
- Bielefeld Meetup May 22nd: 22 May 2013 07:00PM
- Berlin Social Meetup: 15 June 2013 05:00PM
- Bratislava lesswrong meetup III: 20 May 2013 06:30PM
- Brussels meetup: 18 May 2013 01:00PM
- Durham/RTLW HPMoR discussion, ch. 65-68: 18 May 2013 12:30PM
- London Meetup: 26th May: 26 May 2013 02:00PM
- [Moscow] Belief cleaning: 26 May 2013 04:00PM
- Paris Meetup: Sunday, May 26.: 26 May 2013 02:00PM
The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:
- Austin, TX: 18 May 2019 01:30PM
- Seattle-Vancouver Kilomeetup: 18 May 2013 11:54AM
- Vienna meetup #3: 18 May 2013 04:00PM
Locations with regularly scheduled meetups: Austin, Berkeley, Cambridge, MA, Cambridge UK, Madison WI, Melbourne, Mountain View, New York, Ohio, Portland, Salt Lake City, Seattle, Toronto, Vienna, Waterloo, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers.
Weekly LW Meetups
Irregularly scheduled Less Wrong meetups are taking place in:
- Berlin Social Meetup: 15 June 2013 05:00PM
- Bristol meetup: 25 May 2013 03:00PM
- [Chicago] Fermi Estimates in Chicago: 25 May 2013 03:00PM
- London Meetup: 26th May: 26 May 2013 02:00PM
- [Moscow] Belief cleaning: 26 May 2013 04:00PM
- Munich Meetup: 01 June 2013 03:00PM
- Paris Meetup: Sunday, May 26.: 26 May 2013 02:00PM
The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:
Locations with regularly scheduled meetups: Austin, Berkeley, Cambridge, MA, Cambridge UK, Madison WI, Melbourne, Mountain View, New York, Ohio, Portland, Salt Lake City, Seattle, Toronto, Vienna, Waterloo, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers.
Problems with Academia and the Rising Sea
Severe problems with the biomedical research process
GiveWell has recently been investigating ways to improve biomedical research. When I discovered GiveWell's research was shocked by how severe and comprehensive the problems with the field seem to be:
From a conversation with Ferric Fang:
Because scientists have to compete for grants, they spend a very large fraction of their time fundraising, sometimes more than 50% of their working hours. Scientists feel [strong] pressure to optimize their activities for getting tenure and grants, rather than for doing good science.
From a conversation with Elizabeth Iorns:
Researchers are rewarded primarily for publishing papers in prestigious journals such as Nature, Science and Cell. These journals select for papers that report on surprising and unusual findings. Papers that report on unsound research that is apparently exciting are more likely to be published than papers which report on less exciting research that is sound.
There is little post-publication check on the soundness of papers’ findings, because journals, especially prestigious ones, generally don’t publish replications, and there is little funding for performing replications.
[…]
Pharmaceutical companies such as Bayer and Amgen have studied the frequency with which studies are reproducible by trying to reproduce them, and they have found that about 70% of published papers in the areas that they considered don’t reproduce.
[…]
Because many published results are not reproducible, it is difficult for scientists to use the published literature as a basis for deciding what experiments to perform.
[…]
As things stand, the pharmaceutical industry does replications, however, these are generally unpublished. Because a given lab doesn’t know whether other labs have found that a study fails to replicate, labs duplicate a lot of effort.
From a conversation with Ken Witwer:
Dr. Witwer published a study in Clinical Chemistry examining 127 papers that had been published in between July 2011 and April 2012 in journals that ostensibly require that researchers deposit their microarray data. He found that the data was not submitted for almost 60% of papers, and that data for 75% of papers were not in a format suitable for replication.
The above remarks give the impression that the problems are deeply entrenched and mutually reinforcing. On first glance, it seems that while one might be able to make incremental improvements (such as funding a journal that publishes replications), prospects for big improvements are very poor. But I became more hopeful after learning more.
The Rising Sea
The great mathematician Alexander Grothendieck wrote about two approaches to solving a difficult problem:
If you think of a theorem to be proved as a nut to be opened, so as to reach “the nourishing flesh protected by the shell”, then the hammer and chisel principle is: “put the cutting edge of the chisel against the shell and strike hard. If needed, begin again at many different points until the shell cracks—and you are satisfied”.
[…]
I can illustrate the second approach with the same image of a nut to be opened. The first analogy that came to my mind is of immersing the nut in some softening liquid, and why not simply water? From time to time you rub so the liquid penetrates better, and otherwise you let time pass. The shell becomes more flexible through weeks and months—when the time is ripe, hand pressure is enough, the shell opens like a perfectly ripened avocado!
A different image came to me a few weeks ago. The unknown thing to be known appeared to me as some stretch of earth or hard marl, resisting penetration … the sea advances insensibly in silence, nothing seems to happen, nothing moves, the water is so far off you hardly hear it …. yet it finally surrounds the resistant substance.
When a nut seems too hard to crack, it’s wise to think about the second method that Grothendieck describes.
Alternative Metrics
I was encouraged by GiveWell’s subsequent conversations, with David Jay and Jason Priem, which suggest a “rising sea” type solution to the cluster of apparently severe problems with biomedical research.
In brief, the idea is that it may be possible to create online communities and interfaces that can be used to generate measures of how valuable researchers find research outputs, and which could be used for funding and tenure decisions, thereby rewarding producing the research outputs that other researchers find most valuable. If incentives become aligned with producing valuable research, the whole system will shift accordingly, greatly reducing the existing inefficiencies.
From a conversation with Jason Priem
Historically, the academic community has filtered academic outputs for interest by peer review and, more specifically, the prestige of the journals where papers are published. This model is inadequate relative to filtering mechanisms that are now in principle possible using the Internet.
It is now possible to use the web to measure the quality and impact of an academic output via alternative metrics (altmetrics) such as
- How many people downloaded it
- How much it has been discussed on Twitter
- How many websites link to it
- The caliber of the scientists who have recommended it
- How many people have saved it in a reference manager like Mendeley or Zotero
This is similar to how Google generates a list of webpages corresponding to a search term, since you can benefit from PageRank-type algorithms that foreground popular content in an intelligent fashion.
[…]
There’s been a significant amount of interest from funders and administrators in more nuanced and broader measures of researcher impact than their journal publication record. […] Algorithmically generated rankings of researchers’ influence as measured by the altmetrics mentioned previously could be an input into hiring, tenure, promotion, and grant decisions. ImpactStory and other providers of alternative metrics could help researchers’ aggregate their online impact so that they can present good summaries of it to administrators and funders.
From a conversation with David Jay
Commenting systems could potentially be used to create much more useful altmetrics. Such altmetrics could be generated for a scientific output by examining the nature of the comments that scientists make about it, weighting the comments using factors such as the number of upvotes that a comment receives and how distinguished the commenter is.
The metrics generated would be more informative than a journal publication record, because commenters give more specific feedback than the acceptance/rejection of a paper submitted to a given journal does.
[…]
If scientists were to routinely use online commenting systems to discuss scientific outputs, it seems likely that altmetrics generated from them would be strong enough for them to be used for hiring, promotion and grant-making decisions (in conjunction with, or in place of, the traditional metric of journal publication record).
[…]
David Jay envisages a future in which there is [...] A website which collects analytics from other websites so as to aggregate the impact of individual researchers, both for their own information and for use by hiring/promotion/grant committees.
The viability of this approach remains to be seen, but it could work really well, and illustrate a general principle.
About the author: I worked as a research analyst at GiveWell from April 2012 to May 2013. All views expressed here are my own.
Maximizing Financial Utility and Frugality (Formerly: A Rational Financial Planning Overview)
The past few days have seen an increase of chatter concerning retirement and financial planning. One of us is even putting out a prospectus for a rational financial planning sequence. Some others have derided the concept of saving for retirement, as there is a probability of death before that time.
I am of the Extreme Early Retirement group. The idea is to save and invest 60-90% of your income, and you will have enough money to retire within a decade rather than four decades of the normal working career. This requires you to exercise your frugality muscle (such as cutting cable, biking to work, eating out less), but due to hedonistic adaptation, you will come out no less unhappy.
The sequences have already spoken on how spending money does not make us happier (after our basic needs are met). A Rational Financial plan should take this into account, even if a majority of people would not want to consider it.
I am just a beginner, so I linked the two big names in EEA, Mr. Money Mustache and Early Retirement Extreme. You can find their journeys towards financial independence here and here.
ERE is an austerity heavyweight, while MMM lives a pretty luxurious lifestyle, but still spends much less than his former coworkers. He just spends on what is important to him, such as travelling with his family and eating organic food, and not on anything frivolous, such as cable or eating out. He lives very far from a deprived lifestyle which the average person would shy away from. It takes a paradigm shift and some grit, but the people of LessWrong are not the type to reject munchkin ideas because it takes a little bit of mental effort.
If I were to make a compilation of posts for a Rational Financial Planning sequence, it will go as such…
How Little Money you need to Retire ?
Basic Retirement Math
Rationalist Spending
Maximizing Utilons per Dollar
Utilons Free Of Charge
Investing Rationally Basics
These are just the basics. Investment advice is scare, and the above does not talk about many fianacial aspects, such as insurance, children, career choice. The authors do speak about them on their blog’s, but I omitted them for brevity. Read and follow these posts however, and you will be better off than 90% of your peers, and well on the road to Extreme Early Retirement.
[Edit] This idea of cutting your expenses and maximizing your savings obviously do not apply only to early retirement. Other financial goals, such as saving for a house, building up capital for a business, or giving more money to charity all will be more quickly accomplished if you learn to cut excesses from your life. The driving idea is the cost to live is very small, you are not made any happier by spending money on the extras, and you should put this money where it matters to you the most.
Petruchio
View more: Next

Subscribe to RSS Feed
