This is something I've been thinking about for quite some time:
I had an idea for a web-based app for evaluating instrumental rationality techniques, something like Digg or a UserVoice-based forum where techniques get upvoted, downvoted, merged, separated and discussed. However, I don't currently have a solution for the problem of 'impulse upvoting' ("hey, this technique sounds cool, let's upvote it!") -- I don't know how to make the upvotes reflect long-term usefulness of the techniques.
My current best idea regarding the impulse upvote problem is "making techniques pay rent", literally:
Each app user has a fixed small number of homepage slots for techniques. If a technique doesn't work for a user, they can kick it out of the slot and replace it with another promising technique. Also, users can purchase more homepage slots for real-world money. Or the app can go even further: it can be free to download, but each slot will cost the user $0.99 per month.
This way we can rank the techniques based on their "rent time", i.e. the time they spend in their slot, which is only counted for active users of the app.
Many of the techniques I've found on Less Wrong have increased my available time, money, energy, mood... if the only way I could have learned and used the technique was to have paid money for it, I would gladly. If there was a way to pay back, say, 10% of my actual gains from How to Beat Procrastination to Luke to do with as he wishes, I would press that button. Issues include not correctly estimating the counterfactual (without technique X, how well would I really have done? Surely not a complete crash-and-burn... and what were the actual consequences of ...
I recently heard about SIAI's Rationality Minicamp and thought it sounded cool, but for logistical/expense reasons I won't be going to one.
There are probably lots of people who are interested in improving their instrumental rationality, know about and like LessWrong, but haven't read the vast majority of content because there is just so much material, and the practical payoff is uncertain.
It would be cool if it was much easier for people to find the highest ROI material on LessWrong.
My rough idea for how this new instrumental rationality tool might work:
Do you think others would find this useful? Anyone have suggested improvements?