Summary Geodesic is going to use prediction markets to select their projects for MARS 4.0 and we need your help to make the markets run efficiently! Please read through the proposals, and then trade on the markets for the proposals you think might succeed or fail. We intend to choose...
Preamble This post is my attempt to try and organise some thinking about AI alignment in a way that will act as a partial-overview to the core ideas and approaches. It is mostly a review of existing ideas arranged with some light opinions thrown in. I do not expect the...
Introduction This piece seeks to explore an interesting way of defining intelligent systems such that we can theoretically quantify their general intelligence. From this, further tools and ideas for comparing these entities could be developed. The definitions are not meant to be philosophical truths, rather they are meant to be...
UPDATE:
The projects have been chosen! They are:
These markets will be left locked until their individual metrics are resolvable, all other markets for the un-chosen projects will be resolved N/A.
Thank you to everyone who traded on these markets, and special thanks to those who provided feedback about the research projects and the futarchy experiment itself.
Ahh I see what you mean now, thank you for the clarification.
I agree that in general people trying to exploit and Goodhart LW karma would be bad, though I hope the experiment would not contribute to this. Here, post karma is only being used as a measure, not as a target. The mentors and mentees gain nothing beyond what any other person would normally gain by their research project resulting in a highly-upvoted LW post. Predicted future post karma is just being used optimise over research ideas, and the space of ideas itself is very small (in this experiment) and I doubt we'll get any serious Goodharting by selection of them that... (read more)
Thank you for the suggestion!
Thanks for your engagement with the post. I'm not quite sure I understand what you're getting at? Please could you elaborate?
Thank you for partaking!
Your linked experiment looks very interesting, I will give it a read, thank you for the heads up.
Will you randomize (some of) your choices, as dynomight suggests?
We're not going to randomise choices as the symmetry of the sorts of actions being chosen combined with the fact that the market both makes the decision and mentors are trading on the markets (as suggested by Hanson) means we shouldn't suffer any of the weird oddities decision markets can theoretically occasionally suffer from.
Geodesic is going to use prediction markets to select their projects for MARS 4.0 and we need your help to make the markets run efficiently! Please read through the proposals, and then trade on the markets for the proposals you think might succeed or fail. We intend to choose the best proposals in two weeks!
Full proposals are in Google doc linked below, links to markets are in the section "The Projects".
Google Doc (similar content to this post + full proposal overviews).
Manifold post (similar content to this post).
Geodesic is a new AI safety startup focused on research that is impactful for short AGI/ASI timelines. As part of this, we are committed to mentoring... (read 1192 more words →)
I've ended up making another post somewhat to this effect, trying to predict any significant architectural shifts over the next year and a half: https://manifold.markets/Jasonb/significant-advancement-in-frontier
I made a manifold post for this for those who wish to bet on it: https://manifold.markets/JasonBrown/will-a-gpt4-level-efficient-hrm-bas?r=SmFzb25Ccm93bg
Thank you for providing a good introduction and arguments in favour of this research direction. Whilst I strongly agree with the idea of safety pre-training being valuable (and have even considered working on it myself with some collaborators), I think there are several core claims here that are false and that ultimately one should not consider alignment to be solved.
Thank you!
Your post was also very good and I agree with its points. I'll probably edit my post in the near future to reference it along with some of the other good references your post had that I wasn't aware of.
This post is my attempt to try and organise some thinking about AI alignment in a way that will act as a partial-overview to the core ideas and approaches. It is mostly a review of existing ideas arranged with some light opinions thrown in. I do not expect the content to be particularly new to LessWrong readers, the novelty is in trying to organise existing ideas into one cohesive mental framework that can help scaffold thinking about alignment and AI safety broadly, as well as act as a high-level resource for reference and introduction to those who are new to the field. If there is anything you think I missed and would like to see... (read 3264 more words →)
This piece seeks to explore an interesting way of defining intelligent systems such that we can theoretically quantify their general intelligence. From this, further tools and ideas for comparing these entities could be developed. The definitions are not meant to be philosophical truths, rather they are meant to be useful tools that will allow us to analyse and gain insight into these systems and how they relate to one another. At least that's the hope, failing that they can perhaps at least provide some food for thought.
This post is meant to be accessible to non-technical readers so some terms may be explained to a level of detail unnecessary for people familiar with machine learning.
We... (read 3735 more words →)
Another relevant market predicting in what years CoT monitoring will not work: https://manifold.markets/Jasonb/in-what-years-will-cot-monitoring-f?r=SmFzb25i