LESSWRONG
LW

Mau — LessWrong

Replying toAre short timelines actually bad?

I agree with parts of that. I'd also add the following (or I'd be curious why they're not important effects):

Slower takeoff -> warning shots -> improved governance (e.g. through most/all major actors getting clear[er] evidence of risks) -> less pressure to rush
(As OP argued) Shorter timelines -> China has less of a chance to have leading AI companies -> less pressure to rush

More broadly though, maybe we should be using more fine-grained concepts than "shorter timelines" and "slower takeoffs":

The salient effects of "shorter timelines" seem pretty dependent on what the baseline is.
- The point about China seems important if the baseline is 30 years, and not so much if the baseline is 10

Mau3y

Gradient hacking is extremely difficult

Thanks for writing! I agree the factors this post describes make some types of gradient hacking extremely difficult, but I don't see how they make the following approach to gradient hacking extremely difficult.

Suppose that an agent has some trait which gradient descent is trying to push in direction x because the x-ness of that trait contributes to the agent’s high score; and that the agent wants to use gradient hacking to prevent this. Consider three possible strategies that the agent might try to implement, upon noticing that the x-component of the trait has increased [...] [One potential strategy is] Deterministically increasing the extent to which it fails as the x-component increases.

(from here)

This... (read more)

Replying toThe case against AI alignment

Mau3y

The case against AI alignment

Ah sorry, I meant the ideas introduced in this post and this one (though I haven't yet read either closely).

Replying toThe case against AI alignment

Mau3y*

The case against AI alignment

Thanks for posting, but I think these arguments have major oversights. This leaves me more optimistic about the extent to which people will avoid and prevent the horrible misuse you describe.

First, this post seems to overstate the extent to which people tend to value and carry out extreme torture. Maximally cruel torture fortunately seems very rare.

The post asks "How many people have you personally seen who insist on justifying some form of suffering for those they consider undesirable[?]" But "justifying some form of suffering" isn't actually an example of justifying extreme torture.
The post asks, "What society hasn’t had some underclass it wanted to put down in the dirt just to lord power

... (read 456 more words →)

Replying toLet’s think about slowing down AI

Mau3y

Let’s think about slowing down AI

Thanks for writing!

I want to push back a bit on the framing used here. Instead of the framing "slowing down AI," another framing we could use is, "lay the groundwork for slowing down in the future, when extra time is most needed." I prefer this latter framing/emphasis because:

An extra year in which the AI safety field has access to pretty advanced AI capabilities seems much more valuable for the field's progress (say, maybe 10x) than an extra year with current AI capabilities, since the former type of year would give the field much better opportunities to test safety ideas and more clarity about what types of AI systems are relevant.
- One counterargument is

Mau3y*

Let’s think about slowing down AI

Work to spread good knowledge regarding AGI risk / doom stuff among politicians, the general public, etc. [...] Emphasizing “there is a big problem, and more safety research is desperately needed” seems good and is I think uncontroversial.

Nitpick: My impression is that at least some versions of this outreach are very controversial in the community, as suggested by e.g. the lack of mass advocacy efforts. [Edit: "lack of" was an overstatement. But these are still much smaller than they could be.]

Replying toPredicting GPU performance

Mau3y

Predicting GPU performance

It does, thanks! (I had interpreted the claim in the paper as comparing e.g. TPUs to CPUs, since the quote mentions CPUs as the baseline.)

Replying toPredicting GPU performance

Mau3y

Predicting GPU performance

Thanks! To make sure I'm following, does optimization help just by improving utilization?

Replying toPredicting GPU performance

Mau3y

Predicting GPU performance

Sorry, I'm a bit confused. I'm interpreting the 1st and 3rd paragraphs of your response as expressing opposite opinions about the claimed efficiency gains (uncertainty and confidence, respectively), so I think I'm probably misinterpreting part of your response?

Replying toPredicting GPU performance

Mau3y

Predicting GPU performance

This is helpful for something I've been working on - thanks!

I was initially confused about how these results could fit with claims from this paper on AI chips, which emphasizes the importance of factors other than transistor density for AI-specialized chips' performance. But on second thought, the claims seem compatible:

The paper argues that increases in transistor density have (recently) been slow enough for investment in specialized chip design to be practical. But that's compatible with increases in transistor density still being the main driver of performance improvements (since a proportionally small boost that lasts several years could still make specialization profitable).
The paper claims that "AI[-specialized] chips are tens or even thousands of

... (read more)

AI Governance Needs Technical Work

Mau

Summary and Introduction

People who want to improve the trajectory of AI sometimes think their options for object-level work are (i) technical work on AI alignment or (ii) non-technical work on AI governance. But there is a whole other category of options: technical work in AI governance. This is technical work that mainly boosts AI governance interventions, such as norms, regulations, laws, and international agreements that promote positive outcomes from AI. This piece provides a brief overview of some ways to do this work—what they are, why they might be valuable, and what you can do if you’re interested. I discuss:

Engineering technical levers to make AI coordination/regulation enforceable (through hardware engineering, software/ML engineering,

... (read 2108 more words →)

AI Governance Fundamentals - Curriculum and Application

Mau

[Edit: The course has received major updates since this post was made. See here for the most recent course information and curriculum. I've removed the largely outdated information from this post.]

While many people are interested in transformative AI governance, there is currently no scalable introduction to the field that offers substantial breadth, depth, context, accountability, and information about relevant career opportunities. Anecdotally, finding even a few of those things can be tough. Aiming to improve this state of affairs, I’m excited to introduce the AI Governance Fundamentals program. This program seeks to efficiently bring people up to speed on issues in transformative AI governance through an 11-week virtual course. It consists of 8 weeks of readings, facilitated group discussions, speaker sessions, and a 4-week capstone project, for a total of ~2-4 hours per week.