Yesterday on the Weekly Alignment Research Coffee Time call as people were sharing updates on their recent work, Vanessa Kosoy brought up her and Diffractor's post Infra-Bayesian physicalism: a formal theory of naturalized induction which she was interested in getting feedback on.
Vanessa seemed a bit disappointed/frustrated that this post had received no comments yet. I had to agree after learning that it proposes the first formal decision theory realizing naturalized induction, which has been an open problem in alignment for years. It's also been over 5 weeks since she posted it.
So why hasn't anyone commented? Well it's possible folks have just been busy since it's been the end of the year and the holidays. It also may have to do with that the post includes accompanying proofs and invokes some complex-looking maths; such papers take more time to digest and it's only a subset of people on this forum and LessWrong who are capable of doing so.
But it's also possible that the post was forgotten. Even though it may be important, since it's no longer on the front page of the Alignment Forum and hasn't been referenced anywhere else, nobody would think to look at it again unless it happened to come up in one of their searches or they were prompted about it. In other words, it may have just "fallen through the cracks".
This got me wondering about what other posts on the Alignment Forum might have fallen through the cracks. I did a quick search on the forum to see if I could find out. My search criteria was the following:
- A post received substantial upvotes (arbitrarily 20+ points), suggesting that the community thought it was somewhat interesting or valuable
- But it received no comments or pingbacks on AF - It may still have received comments on LessWrong though or comments/pingbacks by the original authors
- It was posted more than 2 weeks ago
- It's an original "researchey" post - I filtered out posts that were event announcements, linkposts, etc.
Here are the posts that came up in my search. I only went through about ~200 posts for this so consider it exemplary rather than exhaustive:
- Infra-Bayesian physicalism: a formal theory of naturalized induction
- Jitters No Evidence of Stupidity in RL
- Sources of intuitions and data on AGI
- How DeepMind's Generally Capable Agents Were Trained
To be sure, I don't expect all of these posts to necessarily be super important and in dire need of comments. There could be other reasons a post met these criteria, for example that people liked it but it was relatively straightforward and self-contained.
But if there's even a 10% chance that a post was forgotten and that its author is blocked or slowed in their research progress by not receiving any feedback, it seems like an easy win for the community for us to periodically resurface such posts. Authors may be too shy or concerned about looking self-promotional to repost their own work.
Do you know of other posts that you think are important and could benefit from feedback but didn't receive any or enough? This can include your own work. Mention them in the comments below.
I expect this problem to only get worse as the Alignment Forum grows. So I wanted to raise this issue and start a conversation about it. I or someone else could periodically do a post like this to help resurface posts that have fallen through the cracks. Or perhaps there are other more systematic ways we could address this problem as the community scales.
Note: This may remind you of a project that's underway to provide peer review for posts on the Alignment Forum. I consider this to be related but distinct from what I'm talking about here. In that project, Adam, Joe and Jérémy are providing in-depth academic-style peer review for select posts on AF.
I think this is great, but here I'm more concerned about potentially promising posts that haven't gotten any feedback at all. My assumption is that a lot of posts could benefit from having at least one person read through and comment based on their initial impressions - and that the community is large enough to provide this - even if we don't have capacity yet to provide them all with full-fledged peer review.
I remember seeing that post and really wanting to look into it further, but the dense, novel math notation made me postpone that until I forgot about it. (We also had a new baby recently, and I'm working on responding to reviewer comments on a paper I'm trying to publish, so that wasn't the only reason.) It could certainly have helped me grasp the gist of their work if someone had posted a comment at the time summarizing what they got out of it.
What if posts could be flagged by authors and/or the community as needing feedback or discussion? This could work something like pinning to the front page, except that the "pinnedness" could decay over time to make room for other posts while getting periodically refreshed.
I don't know what algorithm is used for sorting front page posts (some combination of recency of the post and its comments and the karma score it has received?), but you could an extra term that has this "jump and decay" behavior. Perhaps the more community members flag it for discussion and feedback, the more often its front-page status gets refreshed. And the more comments it receives, the more the coefficient on this term goes to 0.
Thus, its position in the sorted queue of posts would become like that of a typical post once it has actually gotten feedback (or the author has indicated that the feedback is satisfactory). But it would keep coming back to the community's attention automatically until then.