I wrote a quick draft about this a month ago, working through the simple math of this kind of update. I've just put it up.
Another (very weird) counterpoint: you might not see the "swarm coming" because the annexing of our cosmic endowment might look way stranger than the best strategy human minds can come up with.
I remember a safety researcher once mentioned to me that they didn't necessarily expect us to be killed, just contained, while superintelligence takes over the universe. The argument being that it might want to preserve its history (ie. us) to study it, instead of permanently destroying it. This is basically as bad as also killing everyone, because we'd still be impr...
Currently (although less so now than, say, ten years ago), for LW-ish ideas to come out of the mind of a human into a broken, conformist world requires a conjunction of the idea and rugged individualism. And so we see individualism over-represented. What community-flavored/holism-flavored values might come out of generations growing up with these ideas, I wonder?
It's hard to skim
I don't think these are necessarily mutually exclusive, but we might have to work with alternative formats. But that's a good idea anyway. Here's an example for Bostrom's Astronomical Waste.
It's hard to distinguish sincere claims from hyperbolic ones
This is not really a problem if they're being honest around it. I'm recommending being serious about your silliness, not letting go of your seriousness.
All of your favorite writers probably use it already to some degree. Scott Alexander for instance has explici...
It's true that Open Philanthropy's public communication tends toward a cautious, serious tone (and I think there are good reasons for this); but beyond that, I don't think we do much to convey the sort of attitude implied above. [...] We never did any sort of push to have it treated as a fancy report.
The ability to write in a facetious tone is wonderful addition to one's writing toolset, equivalent to the ability to use fewer significant digits. This is a separate feature from the feature of "fun to read" and "irreverent". People routinely mistake formales...
Yes, and if that's bottlenecked by too few people being good filters, why not teach that?
I would guess that a number of smart people would be able to pick up the ability to spot doomed "perpetual motion alignment strategies" if you paid them a good amount to hang around you for a while.
I really worry about this and it has become quite a block. I want to support fragile baby ontologies emerging in me amidst a cacophony of "objective"/"reward"/etc. taken for granted.
Unfortunately, going off and trying to deconfuse the concepts on my own is slow and feedback-impoverished and makes it harder to keep up with current developments.
I think repurposing "roleplay" could work somewhat, with clearly marked entry and exit into a framing. But ontological assumptions absorb so illegibly that deliberate unseeing is extremely hard, at least without being constantly on guard.
Are there other ways that you recommend (from Framestorming or otherwise?)