I have participated in SERI MATS 2.0 in John's stream. Here is some advice based on my experience.

Be Nice 

The AI alignment community is pretty small. If you are an ass, everybody will know that you are an ass. The same holds to a lesser extent for being nice. When I was visiting Edinburgh to attend a talk by David Krueger, there were several people there, that I had first met at Lightcone. When I was visiting Trajan House, the same thing happened. You never know when you might be talking to a grantmaker over dinner.

Epistemic status: I did not actually behave like an ass. I expect this to be true, based on how many people I ran into that I've seen before, in different parts of the world.

Use Lunch and Dinner at Lightcone 

During MATS 2.0 lunch and dinner were both served at Lightcone every day of the week. There were always many cool people around, and the conversations were unusually insightful. My favorite heuristic is to just join whatever conversation John is in. I am pretty sure that at least 15% of the value of SERI MATS came from eating lunch and dinner at Lightcone. Probably much more than that.

Epistemic status: It feels like this was very useful, but it is hard to quantify.

Take care of yourself

At the beginning of SERI MATS, there were many social events (mostly just general Berkeley EA/Rationalist events). They were all happening pretty late. For some reason, I need to sleep 10:30 to 12:00 hours every day or I will be tired. My team was meeting at 10:00 every day. For the first 3 weeks, I was basically sleep-deprived almost every day. John's workshops are pretty great, and being sleep-deprived during them destroyed probably more than 20% of the value. That being said, at least one of the socials was high-value, and it was probably worth the cost.

The worst thing was that I got used to being sleep-deprived. I sleep-deprived myself, even when there were no socials happening. I made similar mistakes with doing sports and eating healthily. Somehow it's hard to keep up all the good habits when you change your environment.

Epistemic status: It's hard to evaluate the counterfactual where I was not sleep-deprived. I estimate I could have gotten 5-35% more value by not making the mistakes I listed.

Learn to detach yourself from your ideas

Check out this comment.

Be Agentic

If something doesn't fit right, try to fix it.

Do you have a crazy idea about how to improve the office? Ask, or implement it yourself (after getting permission)! (The Lightcone ops team is very competent and cool. John had a loft bed in his office when I was there. I am not sure about the situation in the new SERI MATS offices.)

Choose how you spend your time. If you are in a conversation, notice when you would rather do something else. I recommend that you act on this feeling. Get back to work, join that other discussion that seems more interesting, or do whatever else seems higher value. I think being able to do this is great. Building up this skill is probably easier when talking to rationalists. They won't punish you for this kind of initiative.

In general, being agentic seems highly related to making sure that you thought all your high-value thoughts. I recommend sitting down for at least 5 minutes by the clock every day, and trying to come up with high-value directions to think in. The second step is then to always do what you think is best. Which is not easy.

Think about AI alignment from scratch

Reading somebodies work is different from discovering the underlying insights for yourself. Many details will be omitted in a write-up. Especially details on the research process. When I thought about AI alignment from scratch, I was thinking thoughts I had not thought of before. It seems likely that these thoughts occurred to people like Nick Bostrom, but did not make it into e.g. Superintelligence. Or at least I did not get these thoughts out of Superintelligence by just reading it.

It is easy to read someone's work and regurgitate it whenever it seems relevant. You might even be really good at this and impress other people. But that does not mean that you understood all the important details. It certainly does not mean that you understand the underlying ideas as well as the original author.

I recommend any AI alignment researcher think about the problem of how to solve the AI alignment problem from scratch. At least for a couple of hours every month. And while doing that, try hard to not propose solutions, at least initially. Force your mind to not autocomplete your thinking with solutions you have thought about in the past. Or write them down to get them out of your mind. If you are just starting out, I expect that spending more time thinking about the problem from scratch to be valuable.

Epistemic status: Intuitively this seems very important. I have only limited empirical data on this producing insight. Maybe 5 ideas in myself after thinking for a couple of hours.

Get as independent as possible

I expect the following advice will work best in John's stream. As far as I understand John's goal is to create researchers that can discover new fruitful directions on their own and make progress on them. It might be useful to people not in John's stream.

Right now we don't have a single approach that obviously will lead to a technical solution to the AI alignment problem. I expect there to be many promising directions that nobody has thought of so far. So it seems high value to get people to work on research that is orthogonal to existing research agendas. That means people that can think for themselves and can come up with their own research directions are highly valuable.

Epistemic status: I am pretty uncertain to which extent research directions are underexplored. My intuition tells me that it would be pretty bad if everybody would just work on existing agendas.

Attend the events

In my experience, the SERI MATS events (e.g. talks, and workshops), were all pretty good. I did not attend 2 or 3, and in retrospect, it seems like a mistake. I recommend you attend at least the first 10 minutes of every event, and then decide if you want to leave. We had a wide range of researchers give presentations about their work, which was good for getting a sense of what other people are working on.

Focus on social in the beginning

Spending time getting to know the other people around seems valuable. I am mainly thinking about other SERI MATS scholars and people in the Lightcone offices. Doing this, in the beginning, is probably better for obvious reasons. E.g. you will get to know who knows what, so later on you know who might be able to answer a particular question you have.

Epistemic status: I did do this, though I did not really plan it out. I am somewhat uncertain how useful this is, though I am pretty sure it is positive.

New Comment


6 comments, sorted by Click to highlight new comments since:

Solid advice! But forgive me, I'm gonna jump on something basically unrelated to the rest of the post:

For some reason, I need to sleep 10:30 to 12:00 hours every day or I will be tired.

Yikes! I'm not a doctor and I don't intend to pry, but if you weren't already aware, that's pretty deep into probable-pathology territory. I was doing that sort of thing before figuring out mitigations for my sleep disorder. I didn't quite appreciate how unusual my sleep issues were until very late; I could have saved myself a couple of decades of intense discomfort if I had.

I am now diagnosed with sleep apnea and type 2 narcolepsy. CPAP and a Modafinil prescription seem to help pretty well so far. You were the first iirc to point me in that direction, so thank you. Any things that helped you that I did not list?

I've got a fun suite of weird stuff going on[1], so here's a list of sometimes-very-N=1 data:

  1. Napping: I suck at naps. Despite being very tired, I do not fall asleep easily, and if I do fall asleep, it's probably not going to be for just 5-15 minutes. I also tend to wake up with a lot of sleep inertia, so the net effect of naps on alertness across a day tends to be negative. They also tend to destroy my sleep schedule. 
  2. Melatonin: probably the single most noticeable non-stimulant intervention. While I'm by-default very tired all the time, it's still hard to go to sleep. Without mitigation, this usually meant it was nearly impossible to maintain a 24 hour schedule. Melatonin helps a lot with going to sleep and mostly pauses the forward march (unless I mess up).[2]
  3. Light therapy: subtle, but seems to have an effect. It's more obvious when comparing 'effectively being in a cave' with 'being exposed to a large amount of direct sunlight.' I did notice that, when stacked on everything else, the period where I tried light therapy[3] was the first time I was able to intentionally wake up earlier over the course of several days.
  4. Avoiding excessive light near bed: pretty obviously useful. I've used blue-blocking glasses with some effect, though it's definitely better to just not be exposed to too much light in the first place. I reduce monitor brightness to the minimum if I'm on the computer within 3-5 hours of sleep.
  5. Consistent sleep schedule: high impact, if I can manage it. Having my circadian rhythm fall out of entrainment was a significant contributor[4] to my historical sleeping 10-12 hours a day.[5]
  6. Going to bed earlier: conditioning on waking up with no alarm, sleep duration was not correlated with daytime alertness for me according to my sleep logs. Going to bed early enough such that most of my sleep was at night was correlated.[6]
  7. CPAP: Fiddled with one off-prescription a while since I had access to one and it was cheaper than testing for sleep apnea otherwise. No effect.[7]
  8. Nose strips: hard to measure impact on sleep quality, but subjectively nice! My nosetubes are on the small side, I guess.
  9. Changing detergents/pillows: I seem to react to some detergents, softeners, dust, and stuff along those lines. It's very obvious when I don't double-rinse my pillowcases; my nose swells up to uselessness.
  10. Sleeping room temperature: 62-66F is nice. 72F is not nice. 80F+ is torture.[8]
  11. Watercooled beds: I tried products like eight sleep for a while. If you don't have the ability to reduce the air temperature and humidity to ideal levels, it's worth it, but there is a comfort penalty. It doesn't feel like laying on a fresh and pleasantly cool sheet; it's weirdly like laying on somewhat damp sheets that never dry.[9] Way better than nothing, but way worse than a good sleeping environment.[10]
  12. Breathable bedding: surprisingly noticeable. I bought some wirecutter-reviewed cotton percale sheets and a latex mattress. I do like the latex mattress, but I think the sheets have a bigger effect. Don't have data on whether it meaningfully changed sleep quality, but it is nice.
  13. Caffeine: pretty standard. Helps a bit. Not as strong as prescription stimulants at reasonable dosages, can't use it every day without the effect diminishing very noticeably. And without tolerance, consuming it much later than immediately after getting out of bed disrupts my sleep the next night. I tend to drink some coffee in the morning on days where I don't take other stimulants to make the mornings suck less.
  14. Protriptyline: sometimes useful, but a very bad time for me. Got pretty much all the side effects, including the "stop taking and talk to your doctor immediately" kind and uncomfortably close to the "go to a hospital" kind.[11]
  15. Modafinil: alas, no significant effect. Maybe slightly clumsier, maybe slightly longer sleep, maybe slightly more tired. Best guess is that it interfered with my sleep a little bit.
  16. Ritalin: Works! I use a low dose (12.5 mg/day) of the immediate release generic. Pretty short half-life, but that's actually nice for being able to go to sleep. I often cut pills in half to manually allocate alertness more smoothly. I can also elect to just not take it before a plane flight or on days where being snoozey isn't a big problem.
  17. Stimulant juggling/off days: very hard to tell if there's an effect on tolerance with N=1 for non-caffeine stimulants at low therapeutic dosages. I usually do ~5 ritalin days and ~2 caffeine days a week, and I can say that ritalin does still obviously work after several years.[12]
  18. Creatine: I don't notice any sleep/alertness effect, though some people report it. I use it primarily for fitness reasons.[13]
  19. Exercise: hard to measure impact on alertness. Probably some long-term benefit, but if I overdo it on any given day, it's easy to ruin myself. I exercise a bit every day to try to avoid getting obliterated.[14]
  20. Cytomel: this is a weird one that I don't think will be useful to anyone reading this. It turns out that, while my TSH and T4 levels are normal, my untreated T3 levels are very low for still-unclear reasons. I had symptoms of hypothyroidism for decades, but it took until my late 20's to figure out why. Hypothyroidism isn't the same thing as a sleep disorder, but stacking fatigue on a sleep disorder isn't fun.[15]
  21. Meal timing: another weird one. I've always had an unusual tendency towards hypoglycemic symptoms.[16] In its milder form, this comes with severe fatigue that can seem a bit like sleepiness if you squint. As of a few weeks ago with the help of a continuous glucose monitor, I finally confirmed I've got some very wonky blood sugar behavior despite a normal A1C; one notable bit is a pattern of reactive hypoglycemia. I can't avoid hypoglycemia during exercise by e.g. drinking chocolate milk beforehand. I've actually managed to induce mild hypoglycemia by eating a cinnamon roll pancake (and not exercising). Exercising without food actually works a bit better, though I do still have to be careful about the intensity * duration.

I'm probably forgetting some stuff.

  1. ^

    "Idiopathic hypersomnia"-with-a-shrug was the sleep doctor's best guess on the sleep side, plus a weirdo kind of hypothyroidism, plus HEDs, plus something strange going on with blood sugar regulation, plus some other miscellaneous and probably autoimmune related nonsense.

  2. ^

    I tend to take 300 mcg about 2-5 hours before my target bedtime to help with entrainment, then another 300-600 mcg closer to bedtime for the sleepiness promoting effect. 

  3. ^

    In the form of luminette glasses. I wouldn't say they have a great user experience; it's easy to get a headache and the nose doohicky broke almost immediately. That's part of why I didn't keep using them, but I may try again.

  4. ^

    But far from sole!

  5. ^

    While still being tired enough during the day to hallucinate on occasion.

  6. ^

    Implementing this and maintaining sleep consistency functionally requires other interventions. Without melatonin etc., my schedule free-runs mercilessly.

  7. ^

    Given that I was doing this independently, I can't guarantee that Proper Doctor-Supervised CPAP Usage wouldn't do something, but I doubt it. I also monitored myself overnight with a camera. I do a lot of acrobatics, but there was no sign of apneas or otherwise distressed breathing.

  8. ^

    When I was younger, I would frequently ask my parents to drop the thermostat down at night because we lived in one of those climates where the air can kill you if you go outside at the wrong time for too long. They were willing to go down to around 73F at night. My room was east-facing, theirs was west-facing. Unbeknownst to me, there was also a gap between the floor and wall that opened directly into the attic. That space was also uninsulated. Great times.

  9. ^

    It wasn't leaking!

  10. ^

     The cooling is most noticeable at pressure points, so there's a very uneven effect. Parts of your body can feel uncomfortably cold while you're still sweating from the air temperature and humidity.

  11. ^

    The "hmm my heart really isn't working right" issues were bad, but it also included some spooky brain-hijacky mental effects. Genuinely not sure I would have survived six months on it even with total awareness that it was entirely caused by the medication and would stop if I stopped taking it. I had spent some years severely depressed when I was younger, but this was the first time I viscerally understood how a person might opt out... despite being perfectly fine 48 hours earlier.

  12. ^

    I'd say it dropped a little in efficacy in the first week or two, maybe, but not by much, and then leveled out. Does the juggling contribute to this efficacy? No idea. Caffeine and ritalin both have dopaminergic effects, so there's probably a little mutual tolerance on that mechanism, but they do have some differences.

  13. ^

    Effect is still subtle, but creatine is one of the only supplements that has strong evidence that it does anything.

  14. ^

    Beyond the usual health/aesthetic reasons for exercising, I also have to compensate for joint loosey-gooseyness related to doctor-suspected HEDs. Even now, I can easily pull my shoulders out of socket, and last week discovered that (with the help of some post-covid-related joint inflammation), my knees still do the thing where they slip out of alignment mid-step and when I put weight back on them, various bits of soft tissues get crushed. Much better than it used to be; when I was ~18, there were many days where walking was uncomfortable or actively painful due to a combination of ankle, knee, hip, and back pain.

  15. ^

    Interesting note: my first ~8 years of exercise before starting cytomel, including deliberate training for the deadlift, saw me plateau at a 1 rep max on deadlift of... around 155 pounds. (I'm a bit-under-6'4" male. This is very low, like "are you sure you're even exercising" low. I was, in fact, exercising, and sometimes at an excessive level of intensity. I blacked out mid-rep once; do not recommend.)

    Upon starting cytomel, my strength increased by around 30% within 3 months. Each subsequent dosage increase was followed by similar strength increases. Cytomel is not an anabolic steroid and does not have anabolic effects in healthy individuals.

    I'm still no professional powerlifter, but I'm now at least above average within the actively-lifting population of my size. The fact that I "wasted" so many years of exercise was... annoying.

  16. ^

    Going too long without food or doing a little too much exercise is a good way for me to enter a mild grinding hypoglycemic state. More severely, when I went a little too far with intense exercise, I ended up on the floor unable to move while barely holding onto consciousness.

Probably not useful but just in case here are some other medications that are prescribed for narcolepsy (i.e. stuff that makes you not tired):

Solriamfetol is supposed to be more effective than Modafinil. Possibly hard to impossible to get without a prescription. Haven't tried that yet.

Pitolisant is interesting because it has a novel mechanism of action. Possibly impossible to get even with a prescription, as it is super expensive if you don't have the right health insurance. For me, it did not work that well. Only lasted 2-4 hours, and taking multiple doses makes me not be able to sleep.

Great writeup, strong upvoted.

I'd share a similar list of N=1 data I wrote in a facebook post a few years ago but I'm currently unable to access the site due to a content blocker.

I had a similar situation up until my late 30s. A CPAP fixed some of this, and Wellbutrin (aka Buproprion) fixed the rest. Now I'm at 8-9 hours and more rested, rather than 10-14.