Ben Pace

A Sketch of Good Communication

"Often I compare my own Fermi estimates with those of other people, and that’s sort of cool, but what’s way more interesting is when they share what variables and models they used to get to the estimate." – Oliver Habryka, at a model building workshop at FHI in 2016 One question that people in the AI x-risk community often ask is > "By what year do you assign a 50% probability of human-level AGI?" We go back and forth with statements like "Well, I think you're not updating enough on AlphaGo Zero." "But did you know that person X has 50% in 30 years? You should weigh that heavily in your calculations." However, 'timelines' is not the interesting question. The interesting parts are in the causal models behind the estimates. Some possibilities: * Do you have a story about how the brain in fact implements back-propagation, and thus whether current ML techniques have all the key insights? * Do you have a story about the reference class of human brains and monkey brains and evolution, that gives a forecast for how hard intelligence is and as such whether it’s achievable this century? * Do you have a story about the amount of resources flowing into the problem, that uses factors like 'Number of PhDs in ML handed out each year' and 'Amount of GPU available to the average PhD'? Timelines is an area where many people discuss one variable all the time, where in fact the interesting disagreement is much deeper. Regardless of whether our 50% dates are close, when you and I have different models we will often recommend contradictory strategies for reducing x-risk. For example, Eliezer Yudkowsky, Robin Hanson, and Nick Bostrom all have different timelines, but their models tell such different stories about what’s happening in the world that focusing on timelines instead of the broad differences in their overall pictures is a red herring. (If in fact two very different models converge in many places, this is indeed evidence of t

214Mar 31, 2018

Ben Pace

Message

I'm an admin of LessWrong. Here are a few things about me.

I generally feel more hopeful about a situation when I understand it better.
I have signed no contracts nor made any agreements whose existence I cannot mention.
I believe it is good to take responsibility for

...

38771

1096

300

4923

202

14y

Announcing Inkhaven 2: April 2026

I have come to spread the good word: we're doing Inkhaven again, this April 1 – 30. You can apply on the website. The cheers of the first cohort Why are we doing another cohort? Inkhaven activates people as bloggers/writers. We had 41 residents, and all of them completed the...

Jan 1267

A Statistical Analysis of Inkhaven

Okay, we got 41 people to do 30 posts in 30 days. How did it go? How did they like it? Well I just had 36 of them fill out an extensive feedback form. I am devastated with tiredness, but I have to write my last post, so let's take...

Dec 1, 202534

Ben's 10 Tips for Event Feedback Forms

I have made many many feedback forms for events I have run or been a part of. Here are some simple heuristics of mine, that I write for others to learn from and for my collaborators in the future. Most of my events have had between 50 and 500 people...

Nov 30, 202573

A Harried Meeting

An old pub that nobody much visits. An owner who is always in a drugged-out stupor. Background music that never changes. A pub that has remained throughout war and revolution, and a single brick-wall that has not changed all that time. You are supposed to be investigating a murder. A...

Nov 29, 202555

Should you work with evil people?

Epistemic status: Figuring things out. My mind often wanders to what boundaries I ought to maintain between the different parts of my life and people who have variously committed bad acts or have poor character. On the professional side, I think it is a virtue to be able to work...

Nov 28, 202547

Despair, Serenity, Song and Nobility in "Hollow Knight: Silksong"

Fictional universes are oft defined by what their positive affect feels like and what their negative affect feels like. This is the palette that the story is written using. In the videogame Silksong, the negative affect is a hollow, pointless, loneliness. Every character is a bug, crawling through the dirt,...

Nov 27, 202527

Reasons Why I Cannot Sleep

My therapist says I'm more tired today than she's ever seen me. Here are some reasons my brain says I cannot sleep: 1. My boss might lose faith in my ability to manage projects. Something important might go wrong with my project (Inkhaven), and I am the only person on-call...

Nov 26, 202561

Load More (7/206)

LESSWRONG
LW

LESSWRONG
LW

Ben Pace

Ben Pace

Ben Pace

A Sketch of Good Communication

The Costly Coordination Mechanism of Common Knowledge

12 interesting things I learned studying the discovery of nature's laws

7 Vicious Vices of Rationalists

Ben Pace

Announcing Inkhaven 2: April 2026

A Statistical Analysis of Inkhaven

Ben's 10 Tips for Event Feedback Forms

A Harried Meeting

Should you work with evil people?

Despair, Serenity, Song and Nobility in "Hollow Knight: Silksong"

Reasons Why I Cannot Sleep

A Sketch of Good Communication

The Costly Coordination Mechanism of Common Knowledge

12 interesting things I learned studying the discovery of nature's laws

7 Vicious Vices of Rationalists

Announcing Inkhaven 2: April 2026

A Statistical Analysis of Inkhaven

Ben's 10 Tips for Event Feedback Forms

A Harried Meeting

Should you work with evil people?

Despair, Serenity, Song and Nobility in "Hollow Knight: Silksong"

Reasons Why I Cannot Sleep