geoffreymiller — LessWrong

LESSWRONG
LW

Replying toUnless its governance changes, Anthropic is untrustworthy

Unless its governance changes, Anthropic is untrustworthy

TsviBT -- I can't actually follow what you're saying here. Could you please rephrase a little more directly and clearly? I'd like to understand your point. Thanks!

Replying toUnless its governance changes, Anthropic is untrustworthy

geoffreymiller3mo

Unless its governance changes, Anthropic is untrustworthy

Thanks to Mikhail Samin for writing one of the most persuasive and important articles I've read on LessWrong.

I think a lot of the dubious, skeptical, or hostile comments on this post reflect some profound cognitive dissonance.

Rationalists and EAs generally were very supportive of OpenAI at first, and 80,000 Hours encouraged people to work there; then OpenAI betrayed our trust and violated most of the safety commitments that they made. So, we were fooled once.

Then, Rationalists and EAs were generally very supportive of Anthropic, and 80,000 Hours encouraged well-meaning people to work there; then Anthropic betrayed our trust and violated most of the safety commitments they made. So, we were fooled twice. Which... (read more)

-3

Replying toMy talk on AI risks at the National Conservatism conference last week

geoffreymiller5mo

My talk on AI risks at the National Conservatism conference last week

Matrice -- maybe, if it was possible for people to boycott Google/Deepmind, or Microsoft/OpenAI.

But as a practical matter, we can't expect hundreds of millions of people to suddenly switch from gmail to some email alternative, or to switch from Windows to Linux.

It's virtually impossible to organize a successful boycott of all the Big Tech companies that have oligarchic control of people's digital lives, and that are involved in AGI/ASI development.

I still think the key point of leverage of specific, personalized, grassroots social stigmatization of AGI/ASI developers and people closely involved in what they're doing.

(But I could be convinced that Big Tech boycotts might be a useful auxiliary strategy).

Replying toMy talk on AI risks at the National Conservatism conference last week

geoffreymiller5mo

My talk on AI risks at the National Conservatism conference last week

PS the full video of my 15-minute talk was just posted today on the NatCon YouTube channel; here's the link

Replying toMy talk on AI risks at the National Conservatism conference last week

geoffreymiller5mo

My talk on AI risks at the National Conservatism conference last week

Matrice -- for more on the stigmatization strategy, see my EA Forum post from a couple years ago, here

IMHO, a grassroots moral stigmatization campaign by everyone who knows AGI devs would be much more effective that just current users of a company's products boycotting that company.

•••

Replying toMy talk on AI risks at the National Conservatism conference last week

geoffreymiller5mo

My talk on AI risks at the National Conservatism conference last week

Oliver -- that's all very reasonable, and I largely agree.

I've got no problem with people developing narrow, domain-specific AI such as self-driving cars, or smarter matchmaking apps, or suchlike.

I wish there were better terms that could split the AI industry into 'those focused on safe, narrow, non-agentic AI' versus 'those trying to build a Sand God'. It's only the latter who need to be highly stigmatized.

Peace out :)

Replying toMy talk on AI risks at the National Conservatism conference last week

geoffreymiller5mo

My talk on AI risks at the National Conservatism conference last week

Seth - thanks for sharing that link; I hadn't seen it, and I'll read it.

I agree that we should avoid making AI safety either liberal-coded or conservative-coded.

But, we should not hesitate to use different messaging, emphasis, talking points, and verbal styles when addressing liberal or conservative audiences. That's just good persuasion strategy, and it can be done with epistemic and ethical integrity.

Replying toMy talk on AI risks at the National Conservatism conference last week

geoffreymiller5mo

My talk on AI risks at the National Conservatism conference last week

Russell -- I take your point that in most alternative timelines, we would already be dead, decades ago, due to nuclear war, and I often make that point in discussing AI risk, to hammer home the point that humanity does not have any magical 'character armor' that will protect us from extinction, and that nobody is coming to save us if we're dumb enough to develop AGI/ASI.

However, I disagree with the claim that 'our current situation is not one to preserve'. I know people in the military/intelligence communities who work full time on nuclear safety, nuclear non-proliferation, counter-terrorism, etc. There are tens of thousands of smart people across dozens of agencies across... (read more)

Replying toMy talk on AI risks at the National Conservatism conference last week

geoffreymiller5mo

My talk on AI risks at the National Conservatism conference last week

I didn't say that all Rationalists are evil. I do consider myself a Rationalist in many ways, and I've been an active member of LessWrong and EA for years, and have taught several college courses on EA that include Rationalist readings.

What I did say, in relation to my claim that 'they’ve created in a trendy millenarian cult that expects ASIs will fill all their material, social, and spiritual needs', is that 'This is the common denominator among millions of tech bros, AI devs, VCs, Rationalists, and effective accelerationists'.

The 'common denominator' language implies overlap, not total agreement.

And I think there is substantial overlap among these communities -- socially, financially, ethically, geographically.

Many Rationalists have... (read more)

Replying toMy talk on AI risks at the National Conservatism conference last week

geoffreymiller5mo

My talk on AI risks at the National Conservatism conference last week

Yes. And way too much 'AI safety work' boils down to 'getting paid huge amounts by AI companies to do safety-washing & public relations, to kinda sorta help save humanity, but without upsetting my Bay Area roommates & friends & lovers who work on AI capabilities development'.

My talk on AI risks at the National Conservatism conference last week

geoffreymiller

5mo

Lately I’ve been trying to raise awareness of AI risks among American conservatives. Stopping the reckless development of advanced AI agents (including Artificial General Intelligence (AGI) and Artificial Superintelligence (ASI)) should be a human issue, not a partisan issue. Yet most people working in AI safety advocacy lean to the Left politically, and we seem to be ignoring many potential allies on the Right.

This neglect of conservative allies seems suboptimal, given that the Republican Party currently controls all three branches of the U.S. government (including the Presidency, the Supreme Court, and majorities in the House and the Senate). Granted, the pro-AI lobbyists, Big Tech accelerationists, and tech VCs like Andreessen Horowitz have... (read 2955 more words →)

Biomimetic alignment: Alignment between animal genes and animal brains as a model for alignment between humans and AI systems

geoffreymiller

(Note: this was also published on EA Forum, May 26, 2023, here; this is a somewhat rough-and-ready essay, which I'll expand and deepen later; I'm welcome any comments, reactions, and constructive criticism.)

Overview: To clarify how we might align AI systems with humans, it might be helpful to consider how natural selection has aligned animals brains with animals genes, over the last 540 million years of neural evolution.

Introduction

When we face an apparently impossible engineering challenge – such as AI alignment – it’s often useful to check if Nature has already solved it somehow. The field of biomimetic design seeks engineering inspiration from the evolved adaptations in plants or animals that have already work... (read 4708 more words →)

A moral backlash against AI will probably slow down AGI development

geoffreymiller

Note 1 : This was posted to EA Forum on May 31 here, as a submission for the 2023 Open Philanthropy AI Worldviews contest, due May 31, 2023. It addresses Question 1: “What is the probability that AGI is developed by January 1, 2043?”

Note 2: I'm developing a series of several more detailed essays on active moral stigmatization of AI as a strategy for pausing, slowing, or stopping dangerous AI development; so I'd welcome any constructive feedback on this initial essay.

Overview

People tend to view harmful things as evil, and treat them as evil, to minimize their spread and impact. If enough people are hurt, betrayed, or outraged by AI applications, or lose... (read 4140 more words →)

The heritability of human values: A behavior genetic critique of Shard Theory

geoffreymiller

Overview (TL;DR):

Shard Theory is a new approach to understanding the formation of human values, which aims to help solve the problem of how to align advanced AI systems with human values (the ‘AI alignment problem’). Shard Theory has provoked a lot of interest and discussion on LessWrong, AI Alignment Forum, and EA Forum in recent months. However, Shard Theory incorporates a relatively Blank Slate view about the origins of human values that is empirically inconsistent with many studies in behavior genetics indicating that many human values show heritable genetic variation across individuals. I’ll focus in this essay on the empirical claims of Shard Theory, the behavior genetic evidence that challenges those claims,... (read 6042 more words →)

Brain-over-body biases, and the embodied value problem in AI alignment

geoffreymiller

Note: This essay was published here on EA Forum on Sept 21, 2022. The description of the brain-over-body biases in the EA subculture may or may not apply to the Rationalist subculture in LessWrong. This essay builds upon this essay on the heterogeneity of human value types.

Overview

Most AI alignment research focuses on aligning AI systems with the human brain’s stated or revealed preferences. However, human bodies include dozens of organs, hundreds of cell types, and thousands of adaptations that can be viewed as having evolved, implicit, biological values, preferences, and priorities. Evolutionary biology and evolutionary medicine routinely analyze our bodies’ biological goals, fitness interests, and homeostatic mechanisms in terms of how they... (read 7418 more words →)

The heterogeneity of human value types: Implications for AI alignment

geoffreymiller

Note: This essay was originally posted on EA Forum here on Sept 16. I’d welcome comments from LessWrong readers and AI Alignment Forum experts. I have posted some related essays on EA Forum about the importance for AI alignment of considering corporal/body values, religious values, and the diversity of values across individuals. (I might revise those and post them here soon, if people seem interested.)

Overview

I argue that AI ‘alignment with human values’ needs to take more seriously the fact that there are many different types of human values that work in different ways, that have been studied for many decades by diverse behavioral and social sciences, and that need to be explicitly... (read 2973 more words →)

AI alignment with humans... but with which humans?

geoffreymiller

Note: I'm cross-posting this from EA Forum (where I posted it on Sept 8, 2022), in case anybody on LessWrong or the AI Alignment Forum is interested in commenting; note that there were some very helpful suggested readings in the replies to this: https://forum.effectivealtruism.org/posts/DXuwsXsqGq5GtmsB3/ai-alignment-with-humans-but-with-which-humans

Updated tldr: If human aren't aligned with each other (and we aren't, at any level of social organization above the individual), then it'll be very hard for any AI systems to be aligned with 'humans in general'.

Caveat: This post probably raises a naive question; I assume there's at least a 70% chance it's been considered (if not answered) exhaustively elsewhere already; please provide links if so. I've studied evolutionary... (read 775 more words →)