Grant Demaree

Twiblings, four-parent babies and other reproductive technology

Maybe the test case is to delete one chromosome and insert another a chromosome in a fruit fly. Only 4 pairs of chromosomes, already used for genetic modifications with CRISPR

Goal = complete the insertion and still develop a normal fruit fly. I bet this is a fairly inexpensive experiment, within reach of many people on LessWrong

Replying toTwiblings, four-parent babies and other reproductive technology

Grant Demaree3y*

Twiblings, four-parent babies and other reproductive technology

Chromosome selection seems like the most consequential idea here if it's possible

Is it possible now, even in animals? Can you isolate chromosomes without damaging them and assemble them into a viable nucleus?

Edit: also -- strong upvoted because I want to see more of this on LW. Not directly AI but massively affects the gameboard

Replying toWould it be good or bad for the US military to get involved in AI risk?

Would it be good or bad for the US military to get involved in AI risk?

My model of "steering" the military is a little different from that It's over a thousand partially autonomous headquarters, which each have their own interests. The right hand usually doesn't know what the left is doing

Of the thousand+ headquarters, there's probably 10 that have the necessary legitimacy and can get the necessary resources. Winning over any one of the 10 is a sufficient condition to getting the results I described above

In other words, you don't have to steer the whole ship. Just a small part of it. I bet that can be done in 6 months

Replying toWould it be good or bad for the US military to get involved in AI risk?

Would it be good or bad for the US military to get involved in AI risk?

I don't agree, because a world of misaligned AI is known to be really bad. Whereas a world of AI successfully aligned by some opposing faction probably has a lot in common with your own values

Extreme case: ISIS successfully builds the first aligned AI and locks in its values. This is bad, but it's way better than misaligned AI. ISIS want to turn the world into an idealized 7th Century Middle East, which is a pretty nice place compared to much of human history. There's still a lot in common with your own values

-4

Replying toWould it be good or bad for the US military to get involved in AI risk?

Would it be good or bad for the US military to get involved in AI risk?

I bet that's true

But it doesn't seem sufficient to settle the issue. A world where aligning/slowing AI is a major US priority, which China sometimes supports in exchange for policy concessions sounds like a massive improvement over today's world

The theory of impact here is that there's a lot of policy actions to slow down AI, but they're bottlenecked on legitimacy. The US military could provide legitimacy

They might also help alignment, if the right person is in charge and has a lot of resources. But even if 100% their alignment research is noise that doesn't advance the field, military involvement could be a huge net positive

So the real question is:

Is the theory of impact plausible
Are their big risks that mean this does more harm than good

Replying toExtreme risk neutrality isn't always wrong

Because maximizing the geometric rate of return, irrespective of the risk of ruin, doesn't reflect most peoples' true preferences

In the scenario above with the red and blue lines, the full Kelly has a 9.3% chance of losing at least half your money, but the .4 Kelly only has a 0.58% chance of getting an outcome at least that bad

Replying toExtreme risk neutrality isn't always wrong

Would it be good or bad for the US military to get involved in AI risk?

I agree. I think this basically resolves the issue. Once you've added a bunch of caveats:

The bet is mind-bogglingly favorable. More like the million-to-one, and less like the 51% doubling
The bet reflects the preferences of most of the world. It's not a unilateral action
You're very confident that the results will actually happen (we have good reason to believe that the new Earths will definitely be created)

Then it's actually fine to take the bet. At that point, our natural aversion is based on our inability to comprehend the vast scale of a million Earths. I still want to say no, but I'd probably be a yes at reflective equilibrium

Therefore... there's not much of a dilemma anymore

Meant as a neutral question. I'm not sure whether this would be good or bad on net:

Suppose key elements of the US military took x-risk from misaligned strong AI very seriously. Specifically, I mean:

Key scientists at the Defense Threat Reduction Agency. They have a giant budget (~$3B/year) and are literally responsible for x-risks. Current portfolio is focused on nuclear risks with some biosecurity
Influential policy folks at the Office of the Undersecretary of Defense for Policy. Think dignified career civil servants, 2 levels below the most senior political appointees
Commander's Initiative Group at USSTRATCOM. Folks who have the commander's ear, tend to be well-respected, and have a huge effect on which ideas are taken

... (read 269 more words →)

Replying toExtreme risk neutrality isn't always wrong

It doesn't matter who said an idea. I'd rather just consider each idea on its own merits

Replying toExtreme risk neutrality isn't always wrong

I don't think that solves it. A bounded utility function would stop you from doing infinite doublings, but it still doesn't prevent some finite number of doublings in the million-Earths case

That is, if the first round multiplies Earth a millionfold, then you just have to agree that a million Earths is at least twice as good as one Earth

Matt Yglesias on AI Policy

Risk neutrality is the idea that a 50% chance of $2 is equally valuable as a 100% chance of $1. This is a highly unusual preference, but it sometimes makes sense:

Exampleville is infested with an always-fatal parasite. All 1,024 residents have the worm.
For each gold coin you pay, a pharmacist will deworm one randomly selected resident.
You only have one gold coin, but the pharmacist offers you a bet. Flip a weighted coin, which comes up heads 51% of the time. If heads, you double your money. If tails, you lose it all.
You can play as many times as you like.

It makes sense to be risk neutral here. If you keep playing until... (read 1189 more words →)

Yglesias is a widely read center-left journalist. Co-founder of Vox, ex-NYT. Note the implicit invitation: “I also try to make it clear to people who are closer to the object-level work that I’m interested in writing columns on AI policy if they have ideas for me, but they mostly don’t.”

Full article is ungated on his Substack. Relevant excerpt below:

The typical person’s marginal return on investment for efforts to reduce existential risk from misaligned artificial intelligence is going to diminish at an incredibly rapid pace. I have written several times that I think this problem is worth taking seriously and that the people working on it should not be dismissed as cranks. I’m

Are there English-speaking meetups in Frankfurt/Munich/Zurich?

I read a lot of rationalist-adjacents. Outside of LessWrong and ACX, I hardly ever see posts on AI risk. Tyler Cowen of Marginal Revolution writes that, "it makes my head hurt" but hasn't engaged with the issue. Even Zvi spends very few posts on AI risk.

This is surprising, and I wonder what to make of it. Why do the folks most exposed to MIRI-style arguments have so little to say about them?

Here's a few possibilities

Some of the writers disagree that AGI is a major near-term threat
It's unusually hard to think and write about AI risk
The best rationalist-adjacent writers don't feel like they have a deep enough understanding to write about AI risk
There's

... (read 2114 more words →)

Has anyone written a reductionist theory of creativity?

I'll be in Stuttgart, Germany for a few months this summer for work. It's about equidistant from Frankfurt, Munich, and Zurich. Do any of these cities have regular meetups where English is spoken, at least in part?

About me -- I run a software startup, which primarily sells to large military headquarters in NATO countries. My main professional interests are x-risk, decision theory, and geopolitics. Personal interests in powerlifting, crossfit, and distance hiking

What would we do if alignment were futile?

That is, explain creativity from more fundamental building blocks

Is Functional Decision Theory still an active area of research?

This piece, which predates ChatGPT, is no longer endorsed by its author.

Eliezer's recent discussion on AGI alignment is not optimistic.

I consider the present gameboard to look incredibly grim... We can hope there's a miracle that violates some aspect of my background model, and we can try to prepare for that unknown miracle

For this post, instead of debating Eliezer's model, I want to pretend it's true. Let's imagine we've all seen satisfactory evidence for the following:

AGI is likely to be developed soon*
Alignment is a Hard Problem. Current research is nowhere close to solving it, and this is unlikely to change by the time AGI is developed
Therefore, when AGI is first developed, it will

... (read 868 more words →)