All of Benjamin_Todd's Comments + Replies

I'm open to that and felt unsure the post was a good idea after I released it. I had some discussion with him on twitter afterwards, where we smoothed things over a bit: https://x.com/GaryMarcus/status/1888604860523946354

Thanks for the comments! 

I'm especially keen to explore bottlenecks (e.g. another suggestion I saw is that to reach 1 billion a year would require 10x current global lithium production to supply the batteries.)

A factor of 2 for increased difficultly due to processing intensity seems reasonable, and I should have thrown it in. (Though my estimates were to an order of magnitude so this probably won't change the bottom line, and on the other side, many robots will weigh <100kg and some will be non-humanoid.)

Thanks, and fair points!

Note that it you convert only half the car factories, you can still produce 0.5 billion robots per year, so it doesn't change the basics picture that much. (It's all order of magnitude stuff.)

I talk a little about some other estimates - a standard trajectory would be 20-30 years on the long end. ASI enabled could be even faster than 5yr. I agree it would be nice to flesh these out more.

Also agree it would be good to figure out the conversion efficiency better. One factor on the other side is robots involve lighter parts, which apparently makes it easier. Ideally we'd also check for other input factors that could bottleneck production -eg lithium for batteries at over 100m.

That's helpful! Makes me think the all in hardware costs could be off by a factor of 2x. 

I did wonder about maintenance costs, but I figured they wouldn't change the picture too much because I only assume an avg 3 year lifetime for the robot, and figured they wouldn't need a huge amount of maintenance to make it to that point.

Moreover, if there's worthwhile maintenance that extends the lifetime further, then the hardware costs could end up cheaper than my per year estimate. 

I'm also envisioning the costs after a big scale up, and there would be robot repair shops as numerous as car repair, rather than needing to fly in specialists.

That said, I agree it would be interesting to look at how much is spent on car maintenance per year on a car vs. capital costs. (I expect it would be under 10%?)

I'd be happy to put the opening bunch of paragraphs. I was feeling reluctant to cross-post because I often update my articles as I learn more about a topic, and I don't want to keep multiple versions in sync (especially for a lower priority article).

Yes - if anyone reading knows more about manufacturing and could comment on how easy it would be to convert, that would be very helpful.

I also agree it would be interesting to try to do more analysis of how much ASI and robotics could speed up construction of robot factories, by looking at different bottlenecks and how much they could help.

I'm not sure a robot workforce would have a huge effect initially, since there's already a large pool of human workers (though maybe you get some boost by making everything run 24/7). However, at later stages it might become hard to hire enough human workers, while with robots you could keep scaling.

Thanks, great comment.

Seems like we roughly agree on the human-only case. My thinking was that the profit margin would initially be 90-99%, which would create huge economic incentives. Though incentives and coordination were probably stronger in WW2, which could make things slower. Also 10x per year for 5 years sounds like a lot – helpful to point out they didn't quite achieve that in WW2.

With ASI, I agree something like another 5x speed-up sounds plausible.

I agree (1) and (2) are possibilities. However, from a personal planning pov, you should focus preparing for scenarios (i) that might last a long time (ii) where you can affect what happens, since that's where the stakes are.

Scenarios where we all die soon can be mostly be ignored, unless you think they make up most of the probability. (Edit: to be clear it does reduce the value of saving vs. spending, just don't think it's a big effect unless probabilities are high.)

I think (3) is the key way to push back. 

I feel unsure all my preferences are either ... (read more)

True, though I think many people have the intuition that returns diminish faster than log (at least given current tech).

For example, most people think increasing their income from $10k to $20k would do more for their material wellbeing than increasing it from $1bn to $2bn.

I think the key issue is whether new tech makes it easier to buy huge amounts of utility, or that people want to satisfy other preferences beyond material wellbeing (which may have log or even close to linear returns).

I agree the lower bound for output isn't very tight. I'd be very interested to hear other simple rules of thumb you could use to provide a tighter one.

I'll add a note to the section on input tokens that since they don't require KV cache, it's possible to get much closer to the upper bound. 

The opportunities for algorithmic improvements go far beyond the parallelization and mixture of experts methods you mention.

I agree. I'd be very interested in anyone's forecasts for how they might evolve.

I've been working with (very roughly) another ~10x or so improvement in "inference efficiency" by 2030 (or how to measure this and make sure it's independent from other factors). 

By this I mean that if we were able to train a model with 10^26 FLOP this year, achieving a fixed level of learning efficiency, it would require 10X FLOP to generate useful output, while by 2030 it would only require X FLOP to get the same output. 

Thanks that's interesting!

Can I double check, do you think this affects the bottom lines?

The bottom line is supposed to be that FLOP/s vs. FLOP per forward pass can be used as an upper bound, and memory bandwidth vs. model size can be used as an lower bound, and real life efficiency falls somewhere in the middle depending on a many factors (inc. length of KV cache), which I don't try to get into, but is plausibly around 15% of the upper bound for GPT-4 on H100s.

Are you saying that the lower bound for output tokens should maybe be even lower, because the KV cache can be larger than the model weights?

You need to add in the endowments of the colleges as well. The richest college at Cambridge (Trinity) has an endowment of about $1.5bn; whereas the richest college at Oxford has only about $300m.

What are the chances of being a billionaire or getting $30m plus if you go to Harvard rather than an elite uni?

And then what about HBS rather than Harvard?

Agree - Glassdoor is mainly designed to appeal to job seekers. The way they get their data is by only granting access if you reveal your salary. So the salary data ends up tilted towards the people who are seeking jobs.

There's also a sampling problem. Google has ~10,000 engineers, but there's probably only ~100 who earn $1mn+. Large companies normally only have a couple of responses, so even if you sampled everyone randomly, you'd only get ~1 top earner in the sample.

Hi Jonah,

Great posts.

I agree these figures show it's plausible that the value of donations in finance are significantly larger than the direct economic contribution of many jobs, though I see it as highly uncertain. When you're working in highly socially valuable sectors like research or some entrepreneurship, it seems to me that the two are roughly comparable, with big error bars.

However, I don't think this shows it's plausible that earning to give is likely to be the path towards doing the most good. There are many careers that seem to offer influence ov... (read more)

Glassdoor rarely properly includes the top paid employees (those people don't fill out the survey). According to Goldman's own figures, mean compensation per employee (across all employees) is ~$400k. It'll be significantly higher if you're in front office. Your expected earnings from a Goldman job are roughly the mean earnings multiplied by the expected number of years you'll stay at the firm.

I think both research and advocacy (both to governments and among individuals) are highly important, and it's very unclear which is more important at the margin.

It's too simple to say basic research is more important, because advocacy could lead to hugely increased funding for basic research.

We've collated a list of all the approaches that seem to be on the table in the effective altruism community for improving the long-run future. There's some other options, including funding GiveWell and GCRI. This doc also explains a little more of the reasoning behind the approaches. If you like more detail on how 80k might help reduce the risk of extinction, drop me an email at ben@80000hours.org.

In general, I think the question of how best to improve the long-run future is highly uncertain, but has high value of information, so the most important activi... (read more)

Note that Toby is a trustee of CEA and did most of his government consulting due to GWWC, not the FHI, so it's not clear that FHI wins out in terms of influence over government.

Moreover, if your concern is influence over government, CEA could still beat FHI (even if FHI is doing very high level advocacy) by acting as a multiplier on the FHI's efforts (and similar orgs): $1 donated to CEA could lead to more than $1 of financial or human capital delivered to the FHI or similar. I'm not claiming this is happening, but just pointing out that it's too simple to say FHI wins out just because they're doing some really good advocacy.

Disclaimer: I'm the Executive Director of 80,000 Hours, which is part of CEA.

Read the response to poor cause choice and inconsistent attitude toward rigor as "while some EAs might be donating without enough thought, lots of others are investing most of their resources in doing more research"

The monoculture problem is something we often think about how to fix at 80k. We haven't come up with great solutions yet though.

I also argued that the decline in the FB group is not obviously important. And if it's difficult to avoid, but many movements started by a small group of smart people nevertheless go on to achieve a lot, that's also evidence that it's not important.

Hi Ben,

Thanks for the post. I think this is an important discussion. Though I'm also sympathetic to Nick's comment that a significant amount of extra self-reflection is not the most important thing to EA's success.

I just wanted to flag that I think there are attempts to deal with some of these issues, and explain why I think some of these issues are not a problem.

Philosophical difficulties

Effective altruism was founded by philosophers, so I think there's enough effort going into this, including population ethics. (See Nick's comment)

Poor cause choices

There... (read more)

Hi, I'd like to clarify that we prioritise people who are optimising around positive impact, not earning to give. If someone takes earning to give seriously, then we view that as a good indicator, but we speak to lots of people who aren't considering earning to give careers.

I started writing a response, but decided it would be better to summarise my general thoughts on degree choice and post them on our blog. So see our latest thoughts on how to pick a degree.

Insofar as this particular situation goes, I haven't thought about it much, so take this with a pi... (read more)

FYI: There has been a discussion on 80,000 Hours (started by me) about the value of this project and how to maximise it.

Hey John, we discuss this quite a bit in the interview (esp 1st and 2nd questions). Happy to take further more specific questions here though.

People might prefer this pair:

Peter Buffett and Zizek on why philanthropists do more harm than good and Will MacAskill's response on Qz.com

Sorry Luke, I didn't want to bother you so didn't ask, but I should have guessed you would have found this :)

Hi Luke,

This is certainly really important for 80k - it's on our list of strategic considerations to investigate.

We haven't looked into it in depth already, beyond knowledge of some relevant psychology literature (e.g. being primed by images of money has been found to make people more selfish in a couple of (probably dodgy) studies).

We've put a couple of measures in place which seem like they might help to mitigate the types of drift that don't involve updating on new information. First, making a public commitment to make the world a better place in an ef... (read more)

Givewell is effectively attempting to work out which charities most increase human welfare for dollar. So, a charity 'fails' if it becomes clearly less effective than the next best.

Heh almost, but the argument only seems to apply to xrisk. I don't see much reason to think EA movement building is the most effective way to fight global poverty.

that sounds about right!

There must be some, and it we'd certainly like to investigate which areas of industry are the most harmful. But in general, it's pretty hard for a career to result in the deaths of 600 people, which is a lower bound for what you could do with $1m (you could also fund SI for 1-2 years...). The most common harmful careers seem to inflict economic damage, and since the average dollar is spent on stuff which produces much less welfare than malaria nets or catastrophic risk research, you have to do a lot of it to outweigh your donations, like maybe 1-2 orders o... (read more)

That's a fairly common and very interesting question. Carl's got some thoughts on it, which we'll hopefully get written up. It's closely linked to two big and controversial issues: how good is economic growth and how good is technological progress? It's a case of weighing your contribution to that against the extra donations you can make.

Quite a few 80k members are interested in entrepreneurship. We'd definitely like to investigate these kinds of questions. But haven't found anyone yet.

that's a good one. It's going on the list. We have an upcoming series about happiness and career choice. The first (and one of the upcoming posts) are partially relevant. Drethelin's suggesting a good general strategy. If spending's the problem, you could also consider giving up on altruism in that domain, and making a difference in some other way. This is an example of macrooptimisation

Good question. We tend to take our charity evaluation from Givewell (though we've started our evaluation in some areas). So, we wouldn't be able to easily answer this. I don't think we've ever come across a charity which openly states its terms of surrender. What I can say is that the charities that tend to get recommended have a very focused method (e.g. distributing malaria nets) with a measurable outcome (less malaria), so it's pretty obvious if their failing, and that would cause them to lose funding.

It's not all about donating. What's different about us is that we really try to weigh up different career options in terms of how much difference they make. We understand 'making a difference' to mean 'making good stuff happen that wouldn't have happened otherwise.' So, we wouldn't just recommend working for a traditional NGO if someone was going to do that job equally well if you didn't take it. Or if it didn't seem to be particularly cost-effective. In carrying this out, we take an evidence-based approach, paying attention to heuristics and biases. We'd ... (read more)

This seems to essentially be the question 'how can we best reduce xrisk?' We've got people ready to write about this in the fall, if not earlier. As a teaser, it seems like you can make a pretty good argument for EA movement building dominating most of the other approaches.

(Just making this more visible.)

Don't read this until you've already thought about your questions!

But here's what we're already working on:

1. Which people can have the most impact in research careers? When does working in research trump funding research?

   1. How should we factor our own happiness into career decisions? What leads to job satisfaction and how realistic is it to take jobs in industries we're not passionate about?

   2. Among the 'effective altruist' and xrisk organisations, which have the greatest need for more funding or skills of various sorts?

2. W

Apologies - I wasn't intending to hide the fact that I help to run 80k. If I were, hopefully I would have done a better job than using my real name. Point taken about it being a cross posting on the 80k blog, but I did think the content would be of special interest to LWers, and it hasn't been cross posted anywhere else.

This is the first post about 80k on LW by an 80k volunteer/staff member, and like Randaly says, the only two posts in the last 6 months to significantly feature 80k were about arguments for and against professional philanthropy.

Apologies for the 'collage of buzzwords' impression. I didn't include a detailed description of 80k and its purpose, like the THINK post, because I wasn't intending it to be an advert. Rather, I was intending it to be a survey. For this reason I also didn't include much detail about what our existing work is about, hoping not to bia... (read more)

Thanks for the write up Larks. We're currently looking for people to get involved with writing and researching similar articles. If you might be interested, email me: ben@80000hours.org