plex

tl;dr: Mixing goal-directedness into cognitive processes that are working to truth-seek about possible futures tends to undermine both truth-seeking and effective pursuit of your goals. Cleanly separating them has nice properties. [Epistemic Status: Mostly empirically observed rather than rigorously justified, but have seen it many times across different people and...

Your project might be failing without you even knowing it. It’s hard to save the world. If you’re launching a new AI Safety project, this sequence helps you avoid common pitfalls. Your most likely failure modes along the way: You never get started. Entrepreneurship is uncomfortable, and AI Safety is...

tl;dr: Some subagents are more closely managed, which makes them to an extent instruments of the superagent, giving rise to what looks like instrumental/terminal goals. Selection on trust avoids the difficulties that normally come with this, like inability to do open-ended truth-seeking and free-ranging agency. (reply to Richard Ngo on...

You have more context on your ability to make use of funds than fits into a specific numerical ask.[1] You want to give funders good information, and the natural type-signature for this is a utility function over money - how much good you think you can do with different funding...

This reading will have mild spoilers, so feel free to close your ears and gently hum until I raise my hand if you’re strongly averse, but the extracts chosen should more whet than spoil your appetite. The Comet King from the book Scott Alexander's UNSONG is one of the most...

TL;DR: Figure out what needs doing and do it, don't wait on approval from fellowships or jobs. If you... * Have short timelines * Have been struggling to get into a position in AI safety * Are able to self-motivate your efforts * Have a sufficient financial safety net ......

[CW: Retrocausality, omnicide, philosophy] Alternate format: Talk to this post and its sources Three decades ago a strange philosopher was pouring ideas onto paper in a stimulant-fueled frenzy. He wrote that ‘nothing human makes it out of the near-future’ as techno-capital acceleration sheds its biological bootloader and instantiates itself as...