= 27d567bc2d48d9f40e9ccc20ea370b15)
Is CFAR going to market themselves like this?
[at the workshop]:
"Look to the left of you, now to the right of you, now in 12 other directions. Only one of you will have a strong positive effect from this workshop."
I would expect not for a paid workshop! Unlike CFAR's core workshops, which are highly polished and get median 9/10 and 10/10 "are you glad you came" ratings, MSFP
was free and experimental,
produced two new top-notch AI x-risk researchers for MIRI (in my personal judgement as a mathematician, and excluding myself), and
produced several others who were willing hires by the end of the program and who I would totally vote to hire if there were more resources available (in the form of both funding and personnel) to hire them.
Deliberate Grad School
Among my friends interested in rationality, effective altruism, and existential risk reduction, I often hear: "If you want to have a real positive impact on the world, grad school is a waste of time. It's better to use deliberate practice to learn whatever you need instead of working within the confines of an institution."
While I'd agree that grad school will not make you do good for the world, if you're a self-driven person who can spend time in a PhD program deliberately acquiring skills and connections for making a positive difference, I think you can make grad school a highly productive path, perhaps more so than many alternatives. In this post, I want to share some advice that I've been repeating a lot lately for how to do this:
- Find a flexible program. PhD programs in mathematics, statistics, philosophy, and theoretical computer science tend to give you a great deal of free time and flexibility, provided you can pass the various qualifying exams without too much studying. By contrast, sciences like biology and chemistry can require time-consuming laboratory work that you can't always speed through by being clever.
- Choose high-impact topics to learn about. AI safety and existential risk reduction are my favorite examples, but there are others, and I won't spend more time here arguing their case. If you can't make your thesis directly about such a topic, choosing a related more popular topic can give you valuable personal connections, and you can still learn whatever you want during the spare time a flexible program will afford you.
- Teach classes. Grad programs that let you teach undergraduate tutorial classes provide a rare opportunity to practice engaging a non-captive audience. If you just want to work on general presentation skills, maybe you practice on your friends... but your friends already like you. If you want to learn to win over a crowd that isn't particularly interested in you, try teaching calculus! I've found this skill particularly useful when presenting AI safety research that isn't yet mainstream, which requires carefully stepping through arguments that are unfamiliar to the audience.
- Use your freedom to accomplish things. I used my spare time during my PhD program to cofound CFAR, the Center for Applied Rationality. Alumni of our workshops have gone on to do such awesome things as creating the Future of Life Institute and sourcing a $10MM donation from Elon Musk to fund AI safety research. I never would have had the flexibility to volunteer for weeks at a time if I'd been working at a typical 9-to-5 or a startup.
- Organize a graduate seminar. Organizing conferences is critical to getting the word out on important new research, and in fact, running a conference on AI safety in Puerto Rico is how FLI was able to bring so many researchers together on its Open Letter on AI Safety. It's also where Elon Musk made his donation. During grad school, you can get lots of practice organizing research events by running seminars for your fellow grad students. In fact, several of the organizers of the FLI conference were grad students.
- Get exposure to experts. A top 10 US school will have professors around that are world-experts on myriad topics, and you can attend departmental colloquia to expose yourself to the cutting edge of research in fields you're curious about. I regularly attended cognitive science and neuroscience colloquia during my PhD in mathematics, which gave me many perspectives that I found useful working at CFAR.
- Learn how productive researchers get their work done. Grad school surrounds you with researchers, and by getting exposed to how a variety of researchers do their thing, you can pick and choose from their methods and find what works best for you. For example, I learned from my advisor Bernd Sturmfels that, for me, quickly passing a draft back and forth with a coauthor can get a paper written much more quickly than agonizing about each revision before I share it.
- Remember you don't have to stay in academia. If you limit yourself to only doing research that will get you good post-doc offers, you might find you aren't able to focus on what seems highest impact (because often what makes a topic high impact is that it's important and neglected, and if a topic is neglected, it might not be trendy enough land you good post-doc). But since grad school is run by professors, becoming a professor is usually the most salient path forward for most grad students, and you might end up pressuring yourself to follow that standards of that path. When I graduated, I got my top choice of post-doc, but then I decided not to take it and to instead try earning to give as an algorithmic stock trader, and now I'm a research fellow at MIRI. In retrospect, I might have done more valuable work during my PhD itself if I'd decided in advance not to do a typical post-doc.
That's all I have for now. The main sentiment behind most of this, I think, is that you have to be deliberate to get the most out of a PhD program, rather than passively expecting it to make you into anything in particular. Grad school still isn't for everyone, and far from it. But if you were seriously considering it at some point, and "do something more useful" felt like a compelling reason not to go, be sure to first consider the most useful version of grad that you could reliably make for yourself... and then decide whether or not to do it.
Please email me (lastname@thisdomain.com) if you have more ideas for getting the most out of grad school!
In response to
Proposal: Use logical depth relative to human history as objective function for superintelligence
1) Logical depth seems super cool to me, and is perhaps the best way I've seen for quantifying "interestingness" without mistakenly equating it with "unlikeliness" or "incompressibility".
2) Despite this, Manfred's brain-encoding-halting-times example illustrates a way a D(u/h) / D(u) optimized future could be terrible... do you think this future would not obtain, because despite being human-brain-based, would not in fact make much use of being on a human brain? That is, it would have extremely high D(u) and therefore be penalized?
I think it would be easy to rationalize/over-fit our intuitions about this formula to convince ourselves that it matches our intuitions about what is a good future. More realistically, I suspect that our favorite futures have relatively high D(u/h) / D(u) but not the highest value of D(u/h) / D(u).
In response to
Willpower Depletion vs Willpower Distraction
I once asked a room full of about 100 neuroscientists whether willpower depletion was a thing, and there was widespread disagreement with the idea.
In which year did you do the asking?
Great question! It was in the winter of 2013, about a year and a half ago.
Willpower Depletion vs Willpower Distraction
I once asked a room full of about 100 neuroscientists whether willpower depletion was a thing, and there was widespread disagreement with the idea. (A propos, this is a great way to quickly gauge consensus in a field.) Basically, for a while some researchers believed that willpower depletion "is" glucose depletion in the prefrontal cortex, but some more recent experiments have failed to replicate this, e.g. by finding that the mere taste of sugar is enough to "replenish" willpower faster than the time it takes blood to move from the mouth to the brain:
Carbohydrate mouth-rinses activate dopaminergic pathways in the striatum–a region of the brain associated with responses to reward (Kringelbach, 2004)–whereas artificially-sweetened non-carbohydrate mouth-rinses do not (Chambers et al., 2009). Thus, the sensing of carbohydrates in the mouth appears to signal the possibility of reward (i.e., the future availability of additional energy), which could motivate rather than fuel physical effort.-- Molden, D. C. et al, The Motivational versus Metabolic Effects of Carbohydrates on Self-Control. Psychological Science.
Stanford's Carol Dweck and Greg Walden even found that hinting to people that using willpower is energizing might actually make them less depletable:
When we had people read statements that reminded them of the power of willpower like, “Sometimes, working on a strenuous mental task can make you feel energized for further challenging activities,” they kept on working and performing well with no sign of depletion. They made half as many mistakes on a difficult cognitive task as people who read statements about limited willpower. In another study, they scored 15 percent better on I.Q. problems.-- Dweck and Walden, Willpower: It’s in Your Head? New York Times.
While these are all interesting empirical findings, there’s a very similar phenomenon that’s much less debated and which could explain many of these observations, but I think gets too little popular attention in these discussions:
Willpower is distractible.
Indeed, willpower and working memory are both strongly mediated by the dorsolateral prefontal cortex, so “distraction” could just be the two functions funging against one another. To use the terms of Stanovich popularized by Kahneman in Thinking: Fast and Slow, "System 2" can only override so many "System 1" defaults at any given moment.
So what’s going on when people say "willpower depletion"? I’m not sure, but even if willpower depletion is not a thing, the following distracting phenomena clearly are:
- Thirst
- Hunger
- Sleepiness
- Physical fatigue (like from running)
- Physical discomfort (like from sitting)
- That specific-other-thing you want to do
- Anxiety about willpower depletion
- Indignation at being asked for too much by bosses, partners, or experimenters...
... and "willpower depletion" might be nothing more than mental distraction by one of these processes. Perhaps it really is better to think of willpower as power (a rate) than energy (a resource).
If that’s true, then figuring out what processes might be distracting us might be much more useful than saying “I’m out of willpower” and giving up. Maybe try having a sip of water or a bit of food if your diet permits it. Maybe try reading lying down to see if you get nap-ish. Maybe set a timer to remind you to call that friend you keep thinking about.
The last two bullets,
- Anxiety about willpower depletion
- Indignation at being asked for too much by bosses, partners, or experimenters...
are also enough to explain why being told willpower depletion isn’t a thing might reduce the effects typically attributed to it: we might simply be less distracted by anxiety or indignation about doing “too much” willpower-intensive work in a short period of time.
Of course, any speculation about how human minds work in general is prone to the "typical mind fallacy". Maybe my willpower is depletable and yours isn’t. But then that wouldn’t explain why you can cause people to exhibit less willpower depletion by suggesting otherwise. But then again, most published research findings are false. But then again the research on the DLPFC and working memory seems relatively old and well established, and distraction is clearly a thing...
All in all, more of my chips are falling on the hypothesis that willpower “depletion” is often just willpower distraction, and that finding and addressing those distractions is probably a better a strategy than avoiding activities altogether in order to "conserve willpower".
In response to
Voting is like donating thousands of dollars to charity
So I found this paper by Gelman, King, and Boscodarin (1998)
The link is dead. Here's the paper.
Note that the last name of the third author is Boscardin, not Boscodarin.
Thanks, fixed!
CFAR is looking for a videographer for next Wednesday
Hi all, CFAR is looking for a videographer in the Bay Area to shoot and edit a 1-minute video introducing us. Do you know anyone?
If so, please send an email to them and me (critch@rationality.org) that introduces us!
We'll need to shoot the video on Wednesday, Oct 16, or possibly Thursday, Oct 17, and have it edited within about 24 hours.
Thanks for any help tracking someone down!
Sincerely,
--
Critch
In response to
Are coin flips quantum random to my conscious brain-parts?
Let's clear things up a little: you cannot use the category of "quantum random" to actual coin flip, because an object to be truly so it must be in a superposition of at least two different pure states, a situation that with a coin at room temperature has yet to be achieved (and will continue to be so for a very long time). So let's talk about classic randomness from a Bayesian point of view: when you have no prior information that can correlate with the outcome of an event. That's the case with the coin flip (and also with the quantum case, according to many-worlds interpretation).
Since the face landing depends not only on thumb movement but also on the exact starting position and the movement of air molecules, it's surely not possible for you to know all this informations in the beginning to a degree precise enough to deduce the side landing up. In this situation, your "throw the coin" motor impulse and the coin landing are uncorrelated, and so the coin flip is random (from your perspective).
But the degree to which the coin depends on factors you don't control is very low: if you practice enough, you can control the movement of your thumb so that it lands, say, 9 times out of 10 the side you want. In this case you have formed a better model of the coin traveling through the air and you have learned to control your thumb more precisely. In this case the correlation with your motor cortex is much higher and the coin flip is of course no more random.
you cannot use the category of "quantum random" to actual coin flip, because an object to be truly so it must be in a superposition of at least two different pure states, a situation that with a coin at room temperature has yet to be achieved (and will continue to be so for a very long time).
Given the level of subtlety in the question, which gets at the relative nature of superposition, this claim doesn't quite make sense. If I am entangled with a a state that you are not entangled with, it may "be superposed" from your perspective but not from either of my various perspectives.
For example: a projection of the universe can be in state
(you observe NULL)⊗(I observe UP)⊗(photon is spin UP) + (you observe NULL)⊗(I observe DOWN)⊗(photon is spin DOWN) = (you observe NULL)⊗((I observe UP)⊗(photon is spin UP) + (I observe DOWN)⊗(photon is spin DOWN))
The fact that your state factors out means you are disentangled from the joint state of me and the particle, and so together the particle and I are "in a superimposed state" from "your perspective". However, my state does not factor out here; there are (at least) two of me, each observing a different outcome and not a superimposed photon.
Anyway, having cleared that up, I'm not convinced that there is enough mutual information connecting my frontal lobe and the coin for the state of the coin to be entangled with me (i.e. not "in a superposed state") before I observe it. I realize this is testable, e.g., if the state amplitudes of the coin can be forced to have complex arguments differing in a predictable way so as to produce an expected and measurable interference paterns. This is what we have failed to produce at a macroscopic level, and it is this failure that you are talking about when you say
a situation that with a coin at room temperature has yet to be achieved (and will continue to be so for a very long time).
I do not believe I have been shown a convincing empirical test ruling out the possibility that the state is not, from my brain's perspective, in a superposition of vastly many states with amplitudes whose complex arguments are difficult to predict or control well enough to produce clear interference patterns, and half of which are "heads" state and half of which are "tails" states. But I am very ready to be corrected on this, so if anyone can help me out, please do!
Are coin flips quantum random to my conscious brain-parts?
Hello rationality friends! I have a question that I bet some of you have thought about...
I hear lots of people saying that classical coin flips are not "quantum random events", because the outcome is very nearly determined by thumb movement when I flip the coin. More precisely, one can stay that the state of my thumb and the state of the landed coin are strongly entangled, such that, say, 99% of the quantum measure of the coin flips outcomes my post-flip thumb observes all land heads.
First of all, I've never actually seen an order of magnitude estimate to support this claim, and would love it if someone here can provide or link to one!
Second, I'm not sure how strongly entangled my thumb movement is with my subjective experience, i.e., with the parts of my brain that consciously process the decision to flip and the outcome. So even if the coin outcome is almost perfectly determined by my thumb, it might not be almost perfectly determined by my decision to flip the coin.
For example, while the thumb movement happens, a lot of calibration goes on between my thumb, my motor cortex, and my cerebellum (which certainly affects but does not seem to directly process conscious experience), precisely because my motor cortex is unable to send, on its own, a precise and accurate enough signal to my thumb that achieves the flicking motion that we eventually learn to do in order to flip coins. Some of this inability is due to small differences in environmental factors during each flip that the motor cortex does not itself process directly, but is processed by the cerebellum instead. Perhaps some of this inability also comes directly from quantum variation in neuron action potentials being reached, or perhaps some of the aforementioned environmental factors arise from quantum variation.
Anyway, I'm altogether not *that* convinced that the outcome of a coin flip is sufficiently dependent on my decision to flip as to be considered "not a quantum random event" by my conscious brain. Can anyone provide me with some order of magnitude estimates to convince me either way about this? I'd really appreciate it!
ETA: I am not asking if coin flips are "random enough" in some strange, undefined sense. I am actually asking about quantum entanglement here. In particular, when your PFC decides for planning reasons to flip a coin, does the evolution of the wave function produce a world that is in a superposition of states (coin landed heads)⊗(you observed heads) + (coin landed tails)⊗(you observed tails)? Or does a monomial state result, either (coin landed heads)⊗(you observed heads) or (coin landed tails)⊗(you observed tails) depending on the instance?
At present, despite having been told many times that coin flips are not "in superpositions" relative to "us", I'm not convinced that there is enough mutual information connecting my frontal lobe and the coin for the state of the coin to be entangled with me (i.e. not "in a superposed state") before I observe it. I realize this is somewhat testable, e.g., if the state amplitudes of the coin can be forced to have complex arguments differing in a predictable way so as to produce expected and measurable interference patterns. This is what we have failed to produce at a macroscopic level in attempts to produce visible superpositions. But I don't know if we fail to produce messier, less-visibly-self-interfering superpositions, which is why I am still wondering about this...
Any help / links / fermi estimates on this will be greatly appreciated!
= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
Just donated $500 and pledged $6500 more in matching funds (10% of my salary).