What is up with carbon dioxide and cognition? An offer
One or two research groups have published work on carbon dioxide and cognition. The state of the published literature is confusing.
Here is one paper on the topic. The authors investigate a proprietary cognitive benchmark, and experimentally manipulate carbon dioxide levels (without affecting other measures of air quality). They find implausibly large effects from increased carbon dioxide concentrations.
If the reported effects are real and the suggested interpretation is correct, I think it would be a big deal. To put this in perspective, carbon dioxide concentrations in my room vary between 500 and 1500 ppm depending on whether I open the windows. The experiment reports on cognitive effects for moving from 600 and 1000 ppm, and finds significant effects compared to interindividual differences.
I haven't spent much time looking into this (maybe 30 minutes, and another 30 minutes to write this post). I expect that if we spent some time looking into indoor CO2 we could have a much better sense of what was going on, by some combination of better literature review, discussion with experts, looking into the benchmark they used, and just generally thinking about it.
So, here's a proposal:
- If someone looks into this and writes a post that improves our collective understanding of the issue, I will be willing to buy part of an associated certificate of impact, at a price of around $100*N, where N is my own totally made up estimate of how many hours of my own time it would take to produce a similarly useful writeup. I'd buy up to 50% of the certificate at that price.
- Whether or not they want to sell me some of the certificate, on May 1 I'll give a $500 prize to the author of the best publicly-available analysis of the issue. If the best analysis draws heavily on someone else's work, I'll use my discretion: I may split the prize arbitrarily, and may give it to the earlier post even if it is not quite as excellent.
Some clarifications:
- The metric for quality is "how useful it is to Paul." I hope that's a useful proxy for how useful it is in general, but no guarantees. I am generally a pretty skeptical person. I would care a lot about even a modest but well-established effect on performance.
- These don't need to be new analyses, either for the prize or the purchase.
- I reserve the right to resolve all ambiguities arbitrarily, and in the end to do whatever I feel like. But I promise I am generally a nice guy.
- I posted this 2 weeks ago on the EA forum and haven't had serious takers yet.
My research priorities for AI control
I've been thinking about what research projects I should work on, and I've posted my current view. Naturally, I think these are also good projects for other people to work on as well.
Brief summaries of the projects I find most promising:
- Elaborating on apprenticeship learning. Imitating human behavior seems especially promising as a scalable approach to AI control, but there are many outstanding problems.
- Efficiently using human feedback. The limited availability of human feedback may be a serious bottleneck for realistic approaches to AI control.
- Explaining human judgments and disagreements. My preferred approach to AI control requires humans to understand AIs’ plans and beliefs. We don’t know how to solve the analogous problem for humans.
- Designing feedback mechanisms for reinforcement learning. A grab bag of problems, united by a need for proxies of hard-to-optimize, implicit objectives.
Experimental EA funding [crosspost]
Over the course of 2015, we will be distributing $10,000 to completed projects which we believe will have a significant long-term humanitarian impact.
These awards are being made in exchange for certificates of impact. Here's how it works: you tell us about something good you did. We offer you some money. Rather than considering a complicated counterfactual ("How well will this money be spent if I don't take it?"), we encourage you to accept our offer if and only if you would be willing to undo the humanitarian impact of your project in exchange for the money. For more details, see here.
I originally posted this at the EA forum, but it may also be of interest to people here. We are open to funding writing or research on many perennial LW topics (methodological issues, small experiments, lifehacks, useful futurism, etc.).
Why are we buying certificates instead of making grants? Just as market prices help coordinate and incentivize the efficient production of commercial products, they could also help coordinate and incentivize efficient altruism. We also think that paying for performance after the fact has a number of big advantages. Not convinced yet? See a more complete answer.
Applications will include an asking price, the minimum amount of money that would be enough to compensate you for undoing the humanitarian impact of the project. The actual awards will be determined by combining the asking prices with ourimpact assessments in a (truthful) auction. Instead of buying 100% of your project's impact, we'll buy some a fraction less than 50% (at your discretion).
The awards will be made in ten $1,000 rounds, spread over the course of the year. The deadline for the first round is March 25. We'll post the results of each round as they occur. New proposals can be made in between rounds. Once an application is submitted it will be considered in each round unless it is withdrawn.
If you are interested, submit an application here. The application process is designed to be as straightforward as possible. Learn about the kind of work we are most interested in, and see our other restrictions. If you have other questions or comments, contact us atcontact@impactpurchase.org or discuss the project at the effective altruism forum.
impactpurchase.org contains other information about the project, and will describe awards as they are made.
"We" is currently Paul Christiano and Katja Grace. If you are interested in purchasing certificates of impact as part of this effort, we'd love to hear from you.
Recent AI safety work
(Crossposted from ordinary ideas).
I’ve recently been thinking about AI safety, and some of the writeups might be interesting to some LWers:
- Ideas for building useful agents without goals: approval-directed agents, approval-directed bootstrapping, and optimization and goals. I think this line of reasoning is very promising.
- A formalization of one piece of the AI safety challenge: the steering problem. I am eager to see more precise, high-level discussion of AI safety, and I think this article is a helpful step in that direction. Since articulating the steering problem I have become much more optimistic about versions of it being solved in the near term. This mostly means that the steering problem fails to capture the hardest parts of AI safety. But it’s still good news, and I think it may eventually cause some people to revise their understanding of AI safety.
- Some ideas for getting useful work out of self-interested agents, based on arguments: of arguments and wagers, adversarial collaboration [older], and delegating to a mixed crowd. I think these are interesting ideas in an interesting area, but they have a ways to go until they could be useful.
I’m excited about a few possible next steps:
- Under the (highly improbable) assumption that various deep learning architectures could yield human-level performance, could they also predictably yield safe AI? I think we have a good chance of finding a solution---i.e. a design of plausibly safe AI, under roughly the same assumptions needed to get human-level AI---for some possible architectures. This would feel like a big step forward.
- For what capabilities can we solve the steering problem? I had originally assumed none, but I am now interested in trying to apply the ideas from the approval-directed agents post. From easiest to hardest, I think there are natural lines of attack using any of: natural language question answering, precise question answering, sequence prediction. It might even be possible using reinforcement learners (though this would involve different techniques).
- I am very interested in implementing effective debates, and am keen to test some unusual proposals. The connection to AI safety is more impressionistic, but in my mind these techniques are closely linked with approval-directed behavior.
- I’m currently writing up a concrete architecture for approval-directed agents, in order to facilitate clearer discussion about the idea. This kind of work that seems harder to do in advance, but at this point I think it’s mostly an exposition problem.
Approval-directed agents
Most concern about AI comes down to the scariness of goal-oriented behavior. A common response to such concerns is “why would we give an AI goals anyway?” I think there are good reasons to expect goal-oriented behavior, and I’ve been on that side of a lot of arguments. But I don’t think the issue is settled, and it might be possible to get better outcomes without them. I flesh out one possible alternative here, based on the dictum "take the action I would like best" rather than "achieve the outcome I would like best."
(As an experiment I wrote the post on medium, so that it is easier to provide sentence-level feedback, especially feedback on writing or low-level comments.)
Changes to my workflow
About 18 months ago I made a post here on my workflow. I've received a handful of requests for follow-up, so I thought I would make another post detailing changes since then. I expect this post to be less useful than the last one.
For the most part, the overall outline has remained pretty stable and feels very similar to 18 months ago. Things not mentioned below have mostly stayed the same. I believe that the total effect of continued changes have been continued but much smaller improvements, though it is hard to tell (as opposed to the last changes, which were more clearly improvements).
Based on comparing time logging records I seem to now do substantially more work on average, but there are many other changes during this period that could explain the change (including changes in time logging). Changes other than work output are much harder to measure; I feel like they are positive but I wouldn't be surprised if this were an illusion.
Splitting days:
I now regularly divide my day into two halves, and treat the two halves as separate units. I plan each separately and reflect on each separately. I divide them by an hour long period of reflecting on the morning, relaxing for 5-10 minutes, napping for 25-30 minutes, processing my emails, and planning the evening. I find that this generally makes me more productive and happier about the day. Splitting my days is often difficult due to engagements in the middle of the day, and I don't have a good solution to that.
WasteNoTime:
I have longstanding objections to explicitly rationing internet use (since it seems either indicative of a broader problem that should be resolved directly, or else to serve a useful function that would be unwise to remove). That said, I now use the extension WasteNoTime to limit my consumption of blogs, webcomics, facebook, news sites, browser games, etc., to 10 minutes each half-day. This has cut the amount of time I spend browsing the internet from an average of 30-40 minutes to an average of 10-15 minutes. It doesn't seem to have been replaced by lower-quality leisure, but by a combination of work and higher-quality leisure.
Similarly, I turned off the newsfeed in facebook, which I found to improve the quality of my internet time in general (the primary issue was that I would sometimes be distracted by the newsfeed while sending messages over facebook, which wasn't my favorite way to use up wastenotime minutes).
I also tried StayFocusd, but ended up adopting WasteNoTime because of the ability to set limits per half-day (via "At work" and "not at work" timers) rather than per-day. I find that the main upside is cutting off the tail of derping (e.g. getting sucked into a blog comment thread, or looking into a particularly engrossing issue), and for this purpose per half-day timers are much more effective.
Email discipline:
I set gmail to archive all emails on arrival and assign them the special label "In." This lets me to search for emails and compose emails, using the normal gmail interface, without being notified of new arrivals. I process the items with label "in" (typically turning emails into todo items to be processed by the same system that deals with other todo items) at the beginning of each half day. Each night I scan my email quickly for items that require urgent attention.
Todo lists / reminders:
I continue to use todo lists for each half day and for a range of special conditions. I now check these lists at the beginning of each half day rather than before going to bed.
I also maintain a third list of "reminders." These are things that I want to be reminded of periodically, organized by day; each morning I look at the day's reminders and think about them briefly. Each of them is copied and filed under a future day. If I feel like I remember a thing well I file it in far in the future, if I feel like I don't remember it well I file it in the near future.
Over the last month most of these reminders have migrated to be in the form "If X, then Y," e.g. "If I agree to do something for someone, then pause, say `actually I should think about it for a few minutes to make sure I have time,' and set a 5 minute timer that night to think about it more clearly." These are designed to fix problems that I notice when reflecting on the day. This is a recommendation from CFAR folks, which seems to be working well, though is the newest part of the system and least tested.
Isolating "todos":
I now attempt to isolate things that probably need doing, but don't seem maximally important; I aim to do them only on every 5th day, and only during one half-day. If I can't finish them in this time, I will typically delay them 5 days. When they spill over to other days, I try to at least keep them to one half-day or the other. I don't know if this helps, but it feels better to have isolated unproductive-feeling blocks of time rather than scattering it throughout the week.
I don't do this very rigidly. I expect the overall level of discipline I have about it is comparable to or lower than a normal office worker who has a clearer division between their personal time and work time.
Toggl:
I now use Toggl for detailed time tracking. Katja Grace and I experimented with about half a dozen other systems (Harvest, Yast, Klok, Freckle, Lumina, I expect others I'm forgetting) before settling on Toggl. It has a depressing number of flaws, but ends up winning for me by making it very fast to start and switch timers which is probably the most important criterion for me. It also offers reviews that work out well with what I want to look at.
I find the main value adds from detailed time tracking are:
1. Knowing how long I've spent on projects, especially long-term projects. My intuitive estimates are often off by more than a factor of 2, even for things taking 80 hours; this can lead me to significantly underestimate the costs of taking on some kinds of projects, and it can also lead me to think an activity is unproductive instead of productive by overestimating how long I've actually spent on it.
2. Accurate breakdowns of time in a day, which guide efforts at improving my day-to-day routine. They probably also make me feel more motivated about working, and improve focus during work.
Reflection / improvement:
Reflection is now a smaller fraction of my time, down from 10% to 3-5%, based on diminishing returns to finding stuff to improve. Another 3-5% is now redirected into longer-term projects to improve particular aspects of my life (I maintain a list of possible improvements, roughly sorted by goodness). Examples: buying new furniture, improvements to my diet (Holden's powersmoothie is great), improvements to my sleep (low doses of melatonin seem good). At the moment the list of possible improvements is long enough that adding to the list is less valuable than doing things on the list.
I have equivocated a lot about how much of my time should go into this sort of thing. My best guess is the number should be higher.
-Pomodoros:
I don't use pomodoros at all any more. I still have periods of uninterrupted work, often of comparable length, for individual tasks. This change wasn't extremely carefully considered, it mostly just happened. I find explicit time logging (such that I must consciously change the timer before changing tasks) seems to work as a substitute in many cases. I also maintain the habit of writing down candidate distractions and then attending to them later (if at all).
For larger tasks I find that I often prefer longer blocks of unrestricted working time. I continue to use Alinof timer to manage these blocks of uninterrupted work.
-Catch:
Catch disappeared, and I haven't found a replacement that I find comparably useful. (It's also not that high on the list of priorities.) I now just send emails to myself, but I do it much less often.
-Beeminder:
I no longer use beeminder. This again wasn't super-considered, though it was based on a very rough impression of overhead being larger than the short-term gains. I think beeminder was helpful for setting up a number of habits which have persisted (especially with respect to daily routine and regular focused work), and my long-term averages continue to satisfy my old beeminder goals.
Project outlines:
I now organize notes about each project I am working on in a more standardized way, with "Queue of todos," "Current workspace," and "Data" as the three subsections. I'm not thrilled by this system, but it seems to be an improvement over the previous informal arrangement. In particular, having a workspace into which I can easily write thoughts without thinking about where they fit, and only later sorting them into the data section once it's clearer how they fit in, decreases the activation energy of using the system. I now use Toggl rather than maintaining time logs by hand.
Randomized trials:
As described in my last post I tried various randomized trials (esp. of effects of exercise, stimulant use, and sleep on mood, cognitive performance, and productive time). I have found extracting meaningful data from these trials to be extremely difficult, due to straightforward issues with signal vs. noise. There are a number of tests which I still do expect to yield meaningful data, but I've increased my estimates for the expensiveness of useful tests substantially, and they've tended to fall down the priority list. For some things I've just decided to do them without the data, since my best guess is positive in expectation and the data is too expensive to acquire.
Seeking paid help for SPARC logistics
This August CFAR is running the Summer Program on Applied Rationality & Cognition, an academic high school summer program, and we are looking for a part-time, temporary logistics manager for the program. If you are interested in the role and live near Berkeley, CA, please fill out the short form here with additional information.
SPARC will be held at UC Berkeley from August 5 - August 17. The work is full time or more from August 3 - August 17, and about 10 hours per week between the start of the job and August 3. The participants are very talented high school math students.
If you are interested in the work CFAR is doing or in SPARC in particular, this will be a great opportunity to see the work first hand, help out, and meet many of the people involved.
Some examples of logistical tasks that you would help with:
- Ensure that facilities are prepared, help arrange housing, determine the schedule and coordinate with visiting instructors, and coordinate with UC Berkeley conference services
- Talk with students and parents in advance of the program, answer questions, give directions, etc.
- Acquire supplies for the program, print and distribute course materials and schedules
- Arrange liability insurance, medical histories, and releases.
SPARC volunteers and instructors will be available to help with some tasks, especially during the program itself, but you would be the main staff responsible for logistics. We strongly prefer applicants who have experience organizing events and ensuring that they run smoothly. Applicants should be local and able to easily get to the UC Berkeley campus. (They should also be happy to interact with 16-18 year old students and their parents.)
Estimates vs. head-to-head comparisons
(Cross-posted from my blog.)
Summary: when choosing between two options, it’s not always optimal to estimate the value of each option and then pick the better one.
Suppose I am choosing between two actions, X and Y. One way to make my decision is to predict what will happen if I do X and predict what will happen if I do Y, and then pick the option which leads to the outcome that I prefer.
My predictions may be both vague and error-prone, and my value judgments might be very hard or nearly arbitrary. But it seems like I ultimately must make some predictions, and must decide how valuable the different outcomes are. So if I have to evaluate N options, I could do it by evaluating the goodness of each option, and then simply picking the option with the highest value. Right?
Induction; or, the rules and etiquette of reference class tennis
(Cross-posted from rationalaltruist.)
Some disasters (catastrophic climate change, high-energy physics surprises) are so serious that even a small probability (say 1%) of such a disaster would have significant policy implications. Unfortunately, making predictions about such unlikely events is extremely unreliable. This makes it difficult to formally justify assigning such disasters probabilities low enough to be compatible with an intuitive policy response. So we must either reconsider our formal analyses or reconsider our intuitive responses.
Intuitively, even if we don’t have an explicit model for a system, we can reason about it inductively, relying on generalizations from historical data. Indeed, this is necessary for virtually all everyday reasoning. But we need to be much more careful about inductive reasoning if we want to use it to obtain 99%+ confidence. In practice such reasoning tends to hinge on questions like “How much should we trust the historical trend X, given that we today face unprecedented condition Y?”
For example, we might wonder: “should we confidently expect historically amiable conditions on Earth to continue in the face of historically unprecedented environmental disruptions?” Human activity is clearly unprecedented in a certain sense, but so is every event. The climate of Earth has changed many times, and probably never undergone the kind of catastrophic climate shift that could destroy human civilization. Should we infer that one more change, at the hands of humans, is similarly unlikely to cause catastrophe? Are human effects so different that we can’t reason inductively, and must resort to our formal models?
Even when we aren't aiming for confident judgments, it can be tricky to apply induction to very unfamiliar domains. Arguments hinging on inductive reasoning often end up mired in disputes about what "reference class" is appropriate in a given setting. Should we put a future war in the (huge) reference class "wars" and confidently predict a low probability of extinction? Should we put a future war in a very small reference class of "wars between nuclear powers" and rely on analytic reasoning to understand whether extinction is likely or not? Both of these approaches seem problematic: clearly a war today is much more dangerous than a war in 1500, but at the same time historical wars do provide important evidence for reasoning about future wars. Which properties of future war should we expect to be contiguous with historical experience? It is easy to talk at length about this question, but it's not clear what constitutes a compelling argument or what evidence is relevant.
In this post I want to take a stab at this problem. Looking over this post in retrospect, I fear I haven't introduced anything new and the reader will come away disappointed. Nevertheless, I very often see confused discussions of induction, not only in popular discourse (which seems inevitable) but also amongst relatively clever altruists. So: if you think that choosing reference classes is straightforward and uncontroversial then please skip this post. Otherwise, it might be worth noting--before hindsight kicks in--that there is a confusion to be resolved. I'm also going to end up covering some basic facts about statistical reasoning; sorry about that.
(If you want you can skip to the 'Reasoning' section below to see an example of what I'm justifying, before I justify it.)
Applying Occam's Razor
I'll start from the principle that simple generalizations are more likely to be correct---whether they are causal explanations (positing the physics governing atoms to explain our observation of atomic spectra) or logical generalizations (positing the generalization that game of life configurations typically break down into still life, oscillators, and gliders). This is true even though each instance of a logical generalization could, given infinite resources, be predicted in advance from first principles. Such a generalization can nevertheless be useful to an agent without infinite resources, and so logical generalizations should be proposed, considered, and accepted by such agents. I'll call generalizations of both types hypotheses.
So given some data to be explained, I suggest (non-controversially) that we survey the space of possible hypotheses, and select a simple set that explains our observations (assigns them high probability) within the limits of our reasoning power. By "within the limits of our reasoning power" means that we treat something as uncertain whenever we can't figure it out, even if we could predict it in principle. I also suggest that we accept all of the logical implications of hypotheses we accept, and rule out those hypotheses which are internally incoherent or inconsistent with our observations.
We face a tradeoff between the complexity of hypotheses and their explanatory power. This is a standard problem, which is resolved by Bayesian reasoning or any other contemporary statistical paradigm. The obvious approach is to choose a prior probability for each hypothesis, and then to accept a hypothesis which maximizes the product of its prior probability and its likelihood---the probability it assigns to our observations. A natural prior is to give a prior of complexity K a prior probability of exp(-K). This basically corresponds to the probability that a monkey would type that hypothesis by chance alone in some particular reference language. This prior probability depends on the language used to define complexity (the language in which the monkey types).
So given some data, to determine the relative probability of two competing hypotheses, we start from the ratio of their prior probabilities, and then multiply by the ratio of their likelihoods. If we restrict to hypotheses which make predictions "within our means"---if we treat the result of a computation as uncertain when we can't actually compute it---then this calculation is tractable for any particular pair of hypotheses.
Arguing
The above section described how the probability of two proposed hypotheses might be compared. That leaves only the problem of identifying the most likely hypotheses. Here I would like to sidestep that problem by talking about frameworks for arguing, rather than frameworks for reasoning.
In fact the world is already full of humans, whose creativity much exceeds any mechanical procedure I could specify (for now). What I am interested are techniques for using those reasoning powers as an "untrusted" black box to determine what is true. I can trust my brain to come up with some good hypotheses, but I don't trust my brain to make sensible judgments about which of those hypotheses are actually true. When we consider groups rather than individuals this situation becomes more extreme---I often trust someone to come up with a good hypothesis, but I don't have any plausible approach for combining everyone's opinion to come up with a reasonable judgment about what is actually true. So, while it's not as awesome as a mechanical method for determining is what is true, I'm happy to settle for a mechanical method for determining who is right in an argument (or at least righter).
Language dependence
The procedure I specified above is nearly mechanical, but is also language-dependent---it will give different answers when applied with different languages, in which different hypotheses look natural. It is plausible that, for example, the climate skeptic and the environmentalist disagree in part because they think about the world in terms of a different set of concepts. A hypothesis that is natural to one of them might be quite foreign to the other, and might be assigned a correspondingly lower prior probability.
Casually, humans articulate hypotheses in a language that contains simple (and relatively uncontroversial) logical/mathematical structure together with a very rich set of concepts. Those concepts come from a combination of biologically enshrined intuitions, individual experiences and learning, and cultural accumulation. People seem tomostly agree about logic and mathematics, and about the abstract reasoning that their concepts "live in."
One (unworkable) approach to eliminating this language dependence is to work with that simple abstract language without any uniquely human concepts. We can then build up more complicated concepts as part of the hypotheses that depend on those concepts. Given enough data about the world, the extra complexity necessary to accommodate such concepts is much more than counterbalanced by their predictive power. For example, even if we start out with a language that doesn't contain "want," the notion of preferences pulls a lot of predictive weight compared to how complicated it is.
The reason this approach is unworkable is that the network of concepts we use is too complicated for us to make explicit or manipulate formally, and the data we are drawing on (and the logical relationships amongst those data) are too complicated to exhaustively characterize. If a serious argument begins by trying to establish that "human" is a natural category to use when talking about the world, it is probably doomed. In light of this, I recommend a more ad hoc approach.
When two people disagree about the relative complexity of two hypotheses, it must be because that hypothesis is simpler in one of their languages than in the other. In light of the above characterization, this disagreement can be attributed to the appearance of at least one concept which one of them thinks is simple but the other thinks is complex. In such cases, it seems appropriate to pass to the meta level and engage in discussion about the complexity of that concept.
In this meta-level argument, the idealized framework---in which we resort to a language of simple connectives and logical operations, uninformed by human experience, and accept a concept when the explanatory power of that concept exceeds its complexity---can serve as a guideline. We can discuss what this idealized process would recommend and accept that recommendation, even though the idealized process is too complicated to actually carry out.
(Of course, even passing to a simple abstract language does not totally eliminate the problem of language dependence. However it does minimize it, since the divergence between the prior probabilities assigned using two different languages is bounded by the complexity of the procedure for translating between them. For simple languages, in an appropriate sense, the associated translation procedures are not complicated. Therefore the divergence in the associated prior probability judgments is small. We only obtain large divergences when we pass to informal human language, which has accumulated an impressive stock of complexity. There is also the empirical observation that people mostly accept common abstract languages. It is possible that someone could end an argument by declaring "yes, if we accept your formalization of logic then your conclusion follows, but I don't." But I've yet to see that failure mode between two serious thinkers.)
Reasoning
I've described a framework for using induction in arguments; now I'd like to look at a few (very similar) examples to try and illustrate the kind of reasoning that is entailed by this framework.
Will project X succeed?
Suppose that X is an ambitious project whose success would cause some historically unprecedented event E (e.g. the resolution of some new technical problem, perhaps human-level machine intelligence). The skeptic wants to argue "no one has done this before; why are you so special?" What does that argument look like, in the framework I've proposed?
The skeptic cites the observation that E has not happened historically, and proposes the hypothesis "E will never happen," which explains the otherwise surprising data and is fairly simple (it gets all of its complexity from the detailed description of exactly what never happens---if E has a natural explanation then it will not be complicated).
The optimist then has a few possible responses:
- The optimist can "explain away" this supporting evidence by providing a more probable explanation for the observation that E hasn't yet happened. This explanation is unlikely to be simpler than the flat-out denial "E will never happen," but it might nevertheless be more probable if it is supported by its own evidence. For example, the optimist might suggest "no one in the past has wanted to do E," together with "E is unlikely to happen unless someone tries to make it happen." Or the optimistic might argue "E was technologically impossible for almost all of history."
- The optimist can provide a sufficiently strong argument for their own success that they overwhelm the prior probability gap between "E will never happen" and "E will first happen in 2013" (or one of the other hypotheses the optimist suggested in [1]).
- The optimist can argue that E is very likely to happen, so that "E will never happen" is very improbable. This will push the skeptic to propose a different hypothesis like "E is very unlikely each year" or "E won't happen for a while." If the optimist can undermine these hypotheses then the ball is in the skeptic's court again. (But note that the skeptic can't say "you haven't given any reason why E is unlikely.")
- The optimist can argue that "E will never happen" is actually a fairly complex hypothesis, because E itself is a complex event (or its apparent simplicity is illusory). The skeptic would then reply by either defending the simplicity of E or offering an alternative generalization, for example showing that E is a special case of a simpler event E' which has also never occurred, or so on.
- Note: the optimist cannot simply say "Project X has novel characteristic C, and characteristic C seems like it should be useful;" this does not itself weaken the inductive argument, at least not if we accept the framework given in this post. The optimist would have to fit this argument into one of the above frameworks, for example by arguing that "E won't happen unless there is a project with characteristic C" as an alternative explanation for the historical record of non-E.
Of course, even if the optimist successfully disarms the inductive argument against project X's success, there will still be many object level considerations to wade through.
Will the development of technology X lead to unprecedented catastrophe Z?
Suppose that I am concerned about the development of technology X because of the apparent risk of catastrophe Z, which would cause unprecedented damage. In light of that concern I suggest that technology X be developed cautiously. A skeptic might say "society has survived for many years without catastrophe Z. Why should it happen now?" This argument is structurally very similar to the argument above, but I want to go through another example to make it more clear.
The skeptic can point to the fact that Z hasn't happened so far, and argue that the generalization "Z is unlikely to happen in any particular year" explains these observations, shifting the burden of proof to the doomsayer. The doomsayer may retort "the advent of technology X is the first time that catastrophe Z has been technologically feasible" (as above), thereby attempting to explain away the skeptic's evidence. This fits into category [1] above. Now the argument can go in a few directions:
- Suppose it is clear ex ante that no previous technologies could not have caused catastrophe Z, but only because we looked exhaustively at each previous technology and seen that it turns out that those technologies couldn't have caused catastrophe Z. Then the generalization "Z is unlikely" still makes predictions---about the properties that technologies have had. So the doomsayer is not clear yet, but may be able to suggest some more likely explanations, e.g. "no previous technologies have created high energy densities" + "without high energy densities catastrophe Z is impossible." This explains all of the observations equally well, and it may be that "no previous technologies have created high energy densities" is more likely a priori (because it follows from other facts about historical technologies which are necessary to explain other observations).
- If many possible technologies haven't actually been developed (though they have been imagined), then "Z is unlikely" is also making predictions. Namely, it is predicting that some imagined technologies haven't been developed. The doomsayer must therefore explain not only why past technologies have not caused catastrophe Z, but why past imagined technologies that could have caused catastrophe Z did not come to pass. How hard this is depends on how many other technologies looked like they might have been developed but did not (if no other Z-causing technologies have ever looked like they might be developed soon, then that observation also requires explanation. Once we've explained that, the hypotheses "Z is unlikely" is not doing extra predictive work).
- In response to [1] or [2], the skeptic can reject the doomsayer's argument that technology X might cause catastrophe Z, but historical technologies couldn't have. In fact the skeptic doesn't have to completely discredit those arguments, he just needs to show that they are sufficiently uncertain that "Z is unlikely" is making a useful prediction (namely that those uncertain arguments actually worked every time, and Z never happened).
- In response to the skeptic's objections, the doomsayer could also argue give up on arguing that there have been no historical points where catastrophe Z might have ensued, and instead argue that there are only a few historical points where Z might have happened, and thus only a few assumptions necessary to explain how Z never happened historically.
- Note: the doomsayer cannot simply say "Technology X has novel characteristic C, and characteristic C might precipitate disaster Z;" this in itself does not weaken the inductive argument. (The doomsayer can make this argument, but has to overcome the inductive argument, however strong it was.)
Risk-aversion and investment (for altruists)
(Cross-posted from rationalaltruist)
Suppose I hope to use my money to do good some day, but for now I am investing it and aiming to maximize my returns. I face the question: how much risk should I be willing to bear? Should I pursue safe investments, or riskier investments with higher returns?
My knee-jerk response is to say “An altruist should be risk neutral. If you have twice as much money, you can do twice as much good. Sure, there are some diminishing returns, but my own investment is minuscule compared to an entire world full of philanthropists. So in the regime where I am investing, returns are roughly linear.” (I might revise this picture if I thought that I was a very unusual philanthropist, and that few others would invest in the same charitable causes as me—in that case I alone might represent a significant fraction of charitable investment in my causes of choice, so I should expect to personally run into diminishing returns.)
But on closer inspection there is something fishy about this reasoning. I don’t have great data on the responsiveness of charitable giving to market performance, but at the individual level it seems that the elasticity of charitable giving to income is about 1—if I am 50% richer (in one possible world than another), I tend to give 50% more to charity. So in worlds where markets do well, we should expect charities to have more money. If markets (rather, the average investor) do 10% better, I should expect 10% more money to be available for any particular charitable cause, regardless of how many donors it has.
Of course, there is a difference between the market’s performance and my portfolio’s performance—the funds available to a charity depend little on how my portfolio does, just how the average donor’s portfolio does. So what is the relevance of the above result to my own risk aversion? Intuitively it seems that the performance of the market is correlated with the performance of any particular investor, but how tightly?
There is a simple observation / folk theorem that applies to this situation. Suppose two investors’ portfolios are not perfectly correlated. Then (supposing those investors are earning the same returns) each would prefer to trade 1/2 of their portfolio for 1/2 of the others—averaging two imperfectly correlated assets reduces the variance. In an efficient economy, this dynamic ensures that every investor’s risk is well-correlated with the market. Any countercyclical assets will be absorbed into this ur-portfolio and invested in by every investor, thereby having the effect of reducing the variance of market returns.
There are many loopholes in this result, and markets are not perfectly efficient, but it provides an important intuition. If we have risky assets that are uncorrelated with (but have comparable returns to) the market, they will just be used to diversify portfolios and thereby become part of “the market.”
So to first order, the fact that I am a small piece of the charitable donations to a cause shouldn’t matter. My risk is well-correlated with the risk of other investors, and if I lose 10% of my money in a year, other investors will also lose 10% of their money, and less money will be available for charitable giving. This holds regardless of whether a cause has a million donors or just one.
The original question: “how risk averse should I be?” is now a question about the returns to charitable activities at a large scale. Clearly the first charitable donations will go to the best causes. How quickly does the quality of marginal charitable giving decline, as total charitable giving increases? This question is fundamentally specific to a cause. For most causes, there seem to be substantial diminishing returns. Some diseases are much easier to treat than others, some disasters easier to mitigate, etc. etc. However, it is worth keeping in mind the distinction between diminishing returns to money in the long-run and in the short-run. For example, if you have only thought of one good thing to do with $1M, your second million dollars would not do nearly as much good if you had to spend it immediately. But this isn’t because the second million dollars is much less valuable than the first million in the long run, it’s because the second million would be complementary with thinking that you haven’t yet done. In the long run you can spend more time thinking about what to do with $2M now that you have it, and put it to a good use. It still won’t be as good as the first million, but not as much less valuable as it appears in the short run.
The following are some important caveats and postscripts to this basic picture.
Ordinary investors are very risk averse
I originally suspected that altruists should be risk-neutral because they are contributing only a small part to large projects, and therefore face roughly linear returns. By now I’ve explicitly rejected that reasoning, but there is another reason that altruists might be interested in risky investments: ordinary investors appear to be extremely risk averse. Evidence of and explanations for the so-called equity premium puzzle are a bit tricky to untangle, but it looks like there is a big premium on risk, such that risky equities earn annual returns a solid 3% higher than risk-free bonds.
If you have logarithmic utility, and estimate risk using historical data on the variability of equities returns, equities are a slam-dunk over safer investments, with risk-adjusted returns that are nearly twice as good. (This observation is the basis for the equity premium puzzle. The paper I linked suggests that the equity premium is smaller than you might naively estimate from US data, but it is still big enough to constitute a puzzle.) I think logarithmic returns are fairly conservative for altruistic projects (though perhaps not individuals’ consumption), and that for most causes the payoffs are much more risk-neutral than that. So it looks like altruists ought to go for risky investments after all.
Moreover, I suspect (on priors) that altruists tend to invest as cautiously as other investors, and so it makes little sense for an altruistic investor to diversify their portfolio between equities and bonds even if there is a significant risk of collapse in equities (other altruists are doing the diversification for them).
Some risks are uncorrelated with the market
There are some opportunities which are risky but imperfectly correlated with the market (and sometimes nearly independent). For example, if you start or invest in a small company, your payoff will depend on that company’s performance (which is typically quite risky but only weakly correlated with the market). In an idealized market this risk would be added to a larger portfolio of risks, but this often impossible due to moral hazard: if you received a paycheck that was independent of the success of your company, you would not be incentivized to run the company well, or to pick good companies to create or invest in. So no one is willing to sell you insurance in case your startup fails or your investment goes bad. The fact that you have to assume a big dose of risk is an unfortunate side-effect of this incentive scheme (and in a more efficient market we would expect angel investors and start-up founders to purchase more extensive insurance for various contingencies that would scuttle their enterprises but are clearly beyond their control).
To a normal person this risk is terrible, but to an altruist it should be considered a good opportunity (since other entrepreneurs and investors will tend to underprice such opportunities). See a discussion of this here, with some quantitative discussion.
This special case is only possible because the entrepreneur or investor is putting in their own effort, and moral hazard makes it hard to smooth out all of the risk across a larger pool (though VC funds will invest in many startups). You shouldn’t expect to find a similar situation in investments, except when you are providing insight which you trust but the rest of the market does not (thereby preventing you from insuring against your risk).
Prioritizing possible worlds and concentrating investments
Risk-aversion is a special case of a more general phenomenon; a dollar is worth a different amount in different possible worlds. For normal risk-aversion, the issue is how much money you have in different possible worlds. A dollar is worth the most in the worlds where you are poorest. For altruists, the issue is how much money charities have in different possible worlds. A dollar is worth the most in the worlds where the least money is given to charities, and the largest number of attractive interventions go unfunded.
But there are other reasons that money may be more valuable in one possible world than another, which depend on which cause you actually want to support. Money aimed at helping the poor is most valuable in worlds where the developing world is not prospering. Vegan outreach is most useful in worlds where the meat industry is doing well, but vegetarian-friendly memes are prospering. Catastrophic risk mitigation is most valuable in troubled times. And so on. Each of these comparisons suggests an investment strategy; investors who care about cause X would prefer have money in worlds where it can be used to further cause X most efficiently.
Moreover, while ordinary risk averse investors are incentivized to construct a diversified portfolios, altruists have no such incentives. Though they should be concerned with risk, what they are really concerned with is the correlation between their risk and market returns. Thus they are not particularly interested in building a diversified portfolio, and it is particularly cheap for them to concentrate their investment in opportunities which will payoff in the worlds where they can best use money. Of course, this strategy becomes less attractive when very few people are interested in cause X, or when many of the investors interested in cause X are pursuing the same strategy—those investors care about the correlation of their investment returns with each other, and collectively they do want to diversify their investments. If everyone who cares about vegetarianism goes broke in worlds where McDonald’s folds, it is no longer the case that vegetarian dollars are less valuable in those worlds.
Investing for the long haul
I think that altruists concerned about the far future should consider investing and earning market returns for as long as possible, before leveraging a much-increased share of the future economy to further their own interests. How does risk relate to this plan?
It seems most productive to think about the fraction of world wealth which an investor controls, since this quantity should be expected to remain fairly constant regardless of what happens economically (though will hopefully drift upwards as long as the altruist is more patient than the average investor) and ultimately controls how much influence that investor wields. A simple argument suggests that an investor concerned with maximizing their influence ought to maximize the expected fraction of world wealth they control. This means that the value of an extra dollar of investment returns should vary inversely with the total wealth of the world. This means that the investor should act as if they were maximizing the expected log-wealth of the world.
The recommendations for this setting (investing for the long haul) are therefore nearly identical to the earlier setting (investing to give). As in the earlier case, the apparent arguments for maximizing expected returns are faulty because it is bad to be correlated with the market. But nevertheless, the equity premium is large enough that investing in risky assets is still worth it. In fact in this case the issue is even more clear-cut, since there is little uncertainty about how risk-averse we should be when investing for the long haul.
The Kelly Criterion
The Kelly criterion is a simple guideline for gambling / investing. The motivating observation is that, to maximize expected long-run returns, it is best to use a logarithmic utility function (because the total return after N periods is the geometric, rather than arithmetic, return during those periods). If we are directly concerned with logarithmic utility, we don’t need to rely on this argument and should just use the Kelly criterion immediately.
The Kelly criterion recommends splitting your money according to the probability of a payout, rather than concentrating all of your money on the single best bet. (See also probability matching, which has been justified on similar grounds.) In the case of investments, this corresponds to the following strategy (if we assume you have a negligible “edge,” or ability to predict tomorrow’s market prices better than other investors). For each asset, estimate what fraction of the current world’s actual wealth is stored in that asset, and invest that fraction of your bankroll in that asset. As prices change, reallocate your money to maintain the same distribution. (If the value of land doubles while the rest of the economy stagnates, such that you now have twice as large a fraction of your bankroll invest in land, then sell off half of your land). Of course, if other investors are following a similar rule, any price changes will be information about the long-run values of the underlying asset values, but this seems to be far from true in the real world.
Investors who support a rare cause and care about diversifying their own portfolio, should probably pursue something like a Kelly strategy. But as I’ve said before, investors pursuing common causes don’t care about diversifying their portfolios, and instead they should use their portfolio to pull theaggregate investments of philanthropists in line with the Kelly rule investments. This seems to mean going all-in on relatively risky assets.
View more: Next
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d)