-
The world is locked right now in a deadly puzzle, and needs something like a miracle of good thought if it is to have the survival odds one might wish the world to have.
-
Despite all priors and appearances, our little community (the "aspiring rationality" community; the "effective altruist" project; efforts to create an existential win; etc.) has a shot at seriously helping with this puzzle. This sounds like hubris, but it is at this point at least partially a matter of track record.[1]
-
To aid in solving this puzzle, we must probably find a way to think together, accumulatively. We need to think about technical problems in AI safety, but also about the full surrounding context -- everything to do with understanding what the heck kind of a place the world is, such that that kind of place may contain cheat codes and trap doors toward achieving an existential win. We probably also need to think about "ways of thinking" -- both the individual thinking skills, and the community conversational norms, that can cause our puzzle-solving to work better. [2]
-
One feature that is pretty helpful here, is if we somehow maintain a single "conversation", rather than a bunch of people separately having thoughts and sometimes taking inspiration from one another. By "a conversation", I mean a space where people can e.g. reply to one another; rely on shared jargon/shorthand/concepts; build on arguments that have been established in common as probably-valid; point out apparent errors and then have that pointing-out be actually taken into account or else replied-to).
-
One feature that really helps things be "a conversation" in this way, is if there is a single Schelling set of posts/etc. that people (in the relevant community/conversation) are supposed to read, and can be assumed to have read. Less Wrong used to be a such place; right now there is no such place; it seems to me highly desirable to form a new such place if we can.
-
We have lately ceased to have a "single conversation" in this way. Good content is still being produced across these communities, but there is no single locus of conversation, such that if you're in a gathering of e.g. five aspiring rationalists, you can take for granted that of course everyone has read posts such-and-such. There is no one place you can post to, where, if enough people upvote your writing, people will reliably read and respond (rather than ignore), and where others will call them out if they later post reasoning that ignores your evidence. Without such a locus, it is hard for conversation to build in the correct way. (And hard for it to turn into arguments and replies, rather than a series of non sequiturs.)
Thoughts on RyanCarey's problems list, point by point:
Not sure all of them are "problems", exactly. I agree that incentive gradients matter, though.
Comments on the specific "problems":
Insofar as 1 is true, it seems like a genuine and simple bug that is probably worth fixing. Matt Graves is I believe the person to talk to if one has ideas or $ to contribute to this. (Or the Arbital crew, insofar as they're taking suggestions.)
The extent to which this is a bug depends on the extent to which posts are aimed at "going viral" / getting shared. If our aim is intellectual generativity, then we do want to attract the best minds of the internet to come think with us, and that does require sometimes having posts go viral. But it doesn't require optimizing the average post for that; it in fact almost benefits from having most posts exist in the relative quiet of a stable community, a community (ideally) with deep intellectual context with which to digest that particular post, such that one can often speak to that community without worrying about whether one's points will be intelligible or palatable to newcomers.
Insofar as writers expect on a visceral level that "number of shares" is the useful thing... people will be pulling against an incentive gradient when choosing LW over Facebook. Insofar as writers come to expect on a visceral level that “adding to this centralized conversational project” tracks value, and that number of shares (from parties who don’t then join the conversation, and who don’t carry on their own good intellectual work elsewhere) is mostly a distraction or blinking light… the incentive may actually come to feel different.
People do sometimes do what is hard when they perceive it to be useful.
I feel there’s an avoidable part of this, which we should avoid; and then an actually useful part of this, which we should keep (and should endeavor to develop positive affect around — when one accurately perceives the usefulness of a thing, it can sometimes come to feel better). See Sarah’s recent post: On Trying Not To Be Wrong
This seems like a bad sign, though I am not sure what to do about it. I don’t think it’s worth compromising the integrity of our conversation for the sake of outside palatability; cross-posting seems plausible; I’d also like to understand it more.
Yep, message me about this, either here or by email (this username at gmail).