LESSWRONG
LW

All of calebp99's Comments + Replies

2Roko
nice!
Should there be just one western AGI project?
calebp9981

I liked various parts of this post and agree that this is an under-discussed but important topic. I found it a little tricky to understand the information security section. Here are a few disagreements (or possibly just confusions).

A single project might motivate more serious attacks, which are harder to defend against.

  • It might also motivate earlier attacks, such that the single project would have less total time to get security measures into place.\

In general, I think it's more natural to think about how expensive an attack will be and how har... (read more)

Reply
2rosehadshar
Thanks for these questions! Earlier attacks: My thinking here is that centralisation might a) cause China to get serious about stealing the weights sooner, and b) therefore allow less time for building up great infosec. So it would be overall bad for infosec. (It's true the models would be weaker, so stealing the weights earlier might not matter so much. But I don't feel very confident that strong infosec would be in place before the models are dangerous (with or without centralisation)) More attack surface: I am trying to compare multiple projects with a single project. The attack surface of a single project might be bigger if the single project itself is very large. As a toy example, imagine 3 labs with 100 employees each. But then USG centralises everything to beat China and pours loads more resources into AGI development. The centralised project has 1000 staff; the counterfactual was 300 staff spread across 3 projects. China stealing weights: sorry, I agree that it's harder for everyone including China, and that all else equal this disincentivises stealing the weights. But a) China is more competent than other actors, so for a fixed increase in difficulty China will be less disincentivised than other actors, b) China has bigger incentives to steal the weights to begin with, and c) for China in particular there might be incentives that push the other way (centralising could increase race dynamics between the US and China, and potentially reduce China's chances of developing AGI first without stealing the weights), and those might counteract the disincentive. Does that make more sense?
Vote on Interesting Disagreements
calebp99169

Most LWers should rely less on norms of their own (or the LW community's) design, and instead defer to regular societal norms more.

Reply3316
Thoughts on open source AI
calebp99130

@peterbarnett and I quickly looked at summaries for ~20 papers citing Llama 2, and we thought ~8 were neither advantaged nor disadvantaged for capabilities over safety, ~7 were better for safety than capabilities, and ~5 were better for capabilities than safety. For me, this was a small update towards the effects of Llama 2 so far, having been positive.

Reply211
LTFF and EAIF are unusually funding-constrained right now
calebp9910

That's helpful feedback; if others would find donating through every.org helpful (which they can signal by agree-voting with the parent comment), I'd be happy to look into this.

I think we can be very flexible for donations over $30k, so if you're interested in making a donation of that size feel free to dm me and I am sure we can figure something out.

Reply
Summary of and Thoughts on the Hotz/Yudkowsky Debate
calebp9930

On my computer, Ctrl-f finds ~10 cases of Holtz appearing in the main text, e.g. point 4 of the introduction.


> ... This included a few times when Yudkowsky’s response was not fully convincing and there was room for Holtz to go deeper, and I wish he would have in those cases. ...

Reply
6localdeity
Oh man.  My brain generates "Was this fixed with a literal s/Holtz/Hotz/ sed command, as opposed to s/Holtz/Hotz/g ?"  Because it seems that, on lines where the name occurs twice or more, the first instance is correctly spelled and the later instances are (edit: sometimes) not.
Alignment Grantmaking is Funding-Limited Right Now
calebp9960

We are hoping to release a report in the next few weeks giving a run down on our grantmaking over the last year, with some explanations for why we made the grants and some high level reflections on the fund.

Some things that might be useful:
* Fund page where we give more context on the goals of the fund: https://funds.effectivealtruism.org/funds/far-future
* Our old payout reports: https://funds.effectivealtruism.org/funds/far-future#payout-reports
* Our public grants database: https://funds.effectivealtruism.org/grants?fund=Long-Term%2520Future%2520Fund&... (read more)

Reply
Alignment Grantmaking is Funding-Limited Right Now
calebp99357

(Speaking just for the Long-Term Future Fund)

It’s true that the Long-Term Future Fund could use funding right now. We’re working on a bunch of posts, including an explanation of our funding needs, track record over the last year, and some reflections that I hope will be out pretty soon.

I’d probably wait for us to post those if you’re a prospective LTFF donor, as they also have a bunch of relevant updates about the fund.

Reply
M. Y. Zuo130

Is there a compiled list of what the LTFF has accomplished and how that compares to past goals and promises, if any, made to previous donors? 

I know some potential donors that would be more readily convinced if they could see such a comparison and reach out to past donors.

Reply
Training Regime Day 1: What is applied rationality?
calebp9910

There are lots of possible goals. Some people are good at achieving some goals. Performance on most goals that are interesting to me is dependent on the decision making ability of the player (e.g. winning at poker vs being tall).

There is some common thread between being an excellent poker player, a supportive friend and a fantastic cook. Even if the inner decision-making workings seem very different in each one, I think that some people have a mindset that lets them find the appropriate decision-making machinery for each task.

To use a metaphor, whilst some... (read more)

Reply
Some thoughts on vegetarianism and veganism
calebp9950

I often see people say things like it is cheaper to follow a vegan diet than an omnivorous one.

I think that this is trivially false (but probably not very interesting), the set of omnivorous diet includes the set of vegan meals and even if the vegan meals are often cheaper than the nonvegan ones, in my personal experience I often find that I am regularly in situations where it would be cheaper to consume a meal that contains meat or dairy (e.g. at restaurants where most meals and not vegan, or when looking around the reduced section of the supermarket).

The... (read more)

Reply
6karlkeefer
On the general point, as a recently-turned vegan (~1 yr), my spending is roughly the same. Money saved on not buying meat/milk/cheese was basically directly replaced by splurging on expensive stuff like avocados, cashews, faux-cheeses, and fancy salads. All of those are non-essential, but budget wasn't ever my primary motive in choosing foods. The following thoughts are mostly in response to your last claim around market dynamics and the foods people choose. A big part of the observed frequency of meat eating is explained by cultural inertia, esp. with the historical signaling function of meat-eating. For a long long time (and still in rural/poor places) owning animals was a primary store of wealth, and killing them to eat them was a very costly display of your fitness. That kind of signal can be culturally baked-in to various food traditions. Fancy restaurants still play this game, with most of the fanciest and most expensive foods being unusual preparations of hard to acquire or raise meat. Another enormous factor here is subsidies (something like $40b annually in the US subsidize meat & dairy). Meat is sometimes cheaper or comparable in price to replacement vegan foods, but that's not a market outcome. Without those subsidies you'd see a bigger price differential. It's also note-worthy that, proportionally, many meals with meat have mostly vegan ingredients. Things like steak are outliers, and many meals that contain meat aren't mostly meat.
What epistemic hygiene norms should there be?
calebp9910

I've been thinking about how we might improve epistemic hygiene in the EA community (particularly on the forum) this post has been useful and I'm keen to find more content in this space.

Reply