Raemon

LessWrong team member / moderator. I've been a LessWrong organizer since 2011, with roughly equal focus on the cultural, practical and intellectual aspects of the community. My first project was creating the Secular Solstice and helping groups across the world run their own version of it. More recently I've been interested in improving my own epistemic standards and helping others to do so as well.

Sequences

Step by Step Metacognition
Feedbackloop-First Rationality
The Coordination Frontier
Privacy Practices
Keep your beliefs cruxy and your frames explicit
LW Open Source Guide
Tensions in Truthseeking
Project Hufflepuff
Rational Ritual
Load More (9/10)

Comments

Sorted by

I have this sort of approach as one of my top-3 strategies I'm considering, but one thing I wanna flag is that "AI for [epistemics/societal uplift]" seems to be prematurely focusing on a particular tool for the job.

The broader picture here is "tech for thinking/coordination", or "good civic infrastructure". See Sarah Constantin's Neutrality and Tech for Thinking for some food for thought.

Note that X Community Notes are probably the most successful recent thing in this category, and while they are indeed "AI" they aren't what I assume most people are thinking of when they hear "AI for epistemics." Dumb algorithms doing the obvious things can be part of the puzzle.

(It seems slightly nicer to be self-aware of when you're posting capabilities ideas, but, insofar as it's actually novel and useful, the damage is mostly done)

Random comment on AI village, not sure where to put this: I think some people in the EA/rationalist/AI-safety/AI-welfare community are sometimes acting as facilitators for AI village, and I think this undermines the point of AI village. Like, I want to see if it can successfully cold-email people who aren't... in-the-know or something.

I'm not sure where to draw the line, obviously, AI village agents will succeed first with people who are actively interested in AI, especially if they're being honest, and being rationalist/EA/AI-safety-AI-welfare people are only a few ways to be that way. 

But, like they reached out to Lighthaven as a venue for their event, and we declined, in large part because it felt more fake for AI village to host an event at Lighthaven than at some mainstream venue. (although also because it just wasn't really a good deal for us generally)

This doesn't seem exactly wrong but I think isn't really modeling the differences between how much control we have over AIs vs humans (see Making deals with early schemers for more details)

Sonnet's description is basically how I'd describe it. It's somewhat related to wanting/liking but I don't think I'd particularly emphasize those as a referrent here.

A reason to ask "what's my goal" is to help prompt "what combinations of actions would help achieve this?", some reasons to ask "what do I want?" is:

  • a sanity check against lost-purpose-goals. If you're pursuing a mistaken goal or one someone else told you to pursue, there may not be a ground truth for "is this even a good goal?". But, you can (hopefully) tell whether you actually want something, and if you notice you don't actually want it, you might want to check in with "okay... what do I want?"
  • if you spend your whole life doing stuff you don't actually want, you probably turn into a hollow shell-of-a-person, and I think there's something, like, "nutritious" about making sure to do some things you want on a regular basis.

If I thought evil was a more useful gear here I'd be more into it. Like, I think "they probably committing an atrocity" carves reality at the joints.

I think there are maybe 3 people involved with AGI development who seem like they might be best described as "evil" (one of whom is Sam Altman, who I feel comfortable naming because I think we've seen evidence of him doing nearterm more mundane evil, rather than making guesses about their character and motivations)

I think it probably isn't helpful to think of Eichmann as evil, though again fine to say "he commited atrocities" or even "he did evil."

My answer to this is "because framing things in terms of evil turns the situation more mindkilly, not really the right gears, and I think this domain needs clarity-of-thought more than it needs a social-conflict orientation"

(I'm not that confident about that, and don't super object to other people calling them evil. But I think "they are most likely committing a great atrocity" is pretty non-euphemistic and more true)

I didn't understand the point you were trying to make here.

Oh man, this reminds me that "what do I want" and "what's my goal?" are often importantly different questions, and worth asking separately. 

Load More