Way back in April 2018, I announced a Prediction Contest, in which the person who made the best predictions on a bunch of questions on PredictionBook ahead of a 1st July deadline would win a prize after they all resolved in January 2019, which is now. It was a bit...
Summary: Make predictions on all the listed predictions before the end of June 2018, and if your predictions are the best you'll win $200 after they all settle in January. About This is happening! Motivations discussed in the earlier post here, I'm running a prediction contest for the rationalist and...
Summary: I think it is worth running a contest to measure ability to make accurate predictions, and am prepared to put up a prize of $200 for the winner. I need to identify a base of 20-50 propositions, all likely to settle within 2-10 months from now, and would like...
On September 28th, we will be celebrating the decision of Lieutenant Colonel Stanislav Yevgrafovich Petrov on September 26th 1983 to not destroy the world.
Despite receiving system reports of multiple US missile launches, Petrov ignored Soviet protocol and chose not to report honestly to his superiors, instead declaring the launch detection a false alarm. Which later turned out to be the case. Thanks to one decision by one man in a tense, time-pressured moment, the world was not destroyed in thermonuclear flame. We choose to celebrate this anniversary because, if anything is worth celebrating, the survival of the human species is.
We’ll be gathering around a (decidedly less powerful) fire at Willowbrook Farm to... (read 161 more words →)
PredictionBook itself has a bunch more than three participants and functions as an always-running contest for calibration, although it's easy to cheat since it's possible to make and resolve whatever predictions you want. I also participate in GJ Open, which has an eternally ongoing prediction contest. So there's stuff out there where people who want to compete on running score can do so.
The objective of the contest was less to bring such an opportunity into existence as to see if it'd incentivise some people who had been "meaning" to practice prediction-making and not gotten to it yet to do so on on one of the platforms, by offering a kind of "reason... (read more)
It's not a novel algorithm type, just a learning project I did in the process of learning ML frameworks, a fairly simple LSTM + one dense layer, trained on the predictions + resolution of about 60% of the resolved predictions from PredictionBook as of September last year (which doesn't include any of the ones in the contest). The remaining resolved predictions were used for cross-validation or set aside as a test set. An even simpler RNN is only very slightly less good, though.
The details of how the algorithm works are thus somewhat opaque but from observing the way it reacts to input, it seems to lean on the average, weight later in... (read more)
Way back in April 2018, I announced a Prediction Contest, in which the person who made the best predictions on a bunch of questions on PredictionBook ahead of a 1st July deadline would win a prize after they all resolved in January 2019, which is now.
It was a bit of an experiment; I had no idea how many people were up for practicing predictions to try to improve their calibration, and decided to throw a little money and time at giving it a try. And in the spirit of reporting negative experimental results: The answer was 3, all of which I greatly appreciate for their participation. I don't regret running the experiment,... (read more)
On September 22nd, we will be celebrating the decision of Lieutenant Colonel Stanislav Yevgrafovich Petrov on September 26th 1983 to not destroy the world.
Despite receiving system reports of multiple US missile launches, Petrov ignored Soviet protocol and chose not to report honestly to his superiors, instead declaring the launch detection a false alarm. Which later turned out to be the case. Thanks to one decision by one man in a tense, time-pressured moment, the world was not destroyed in thermonuclear flame. We choose to celebrate this anniversary because, if anything is worth celebrating, the survival of the human species is.
We’ll be gathering around a (decidedly less powerful) fire at Willowbrook Farm... (read 159 more words →)
I'm concerned that the described examples of holding individual comments to high epistemic standards don't seem to necessarily apply to top-level posts, or linked content- one reason I think this is bad is that it is hard to precisely critique something which is not in itself precise, or which contains metaphor, or which contains example-but-actually-pointing-at-a-class writing where the class can be construed in various different ways.
Critique of fuzzy intuitions and impressions and feelings often involves fuzzy intuitions and impressions and feelings, I think- and if this stuff is restricted in critique but not in top level content it makes top level content involving fuzzy intuitions and impressions and feelings hard to critique,... (read more)
It might be nice to have a set of twenty EA questions, a set of twenty ongoing-academic-research questions, a set of twenty general tech industry questions, a set of twenty world politics questions for the people who like them maybe, and run multiple contests at some point which refine predictive ability within a particular domain, yeah.
It'd be a tough time to source that many, and I feel that twenty is already about the minimum sample size I'd want to use, and for research questions it'd probably require some crowdsourcing of interesting upcoming experiments to predict on, but particularly if help turns out to be available it'd be worth considering if the smaller thing works.
The usefulness of a model of the particular area was something I considered in choosing between questions, but I had a hard time finding a set of good non-personal questions which had very high value to model. I tried to pick questions which in some way depended on interesting underlying questions-for example, the Tesla one hinges on your ability to predict the performance of a known-to-overpromise entrepreneur in a manner that's more precise than either maximum cynicism or full trust, and the ability to predict ongoing ramp-up of manufacturing of tech facing manufacturing difficulties, both of which I think have value.
World politics are I think the weakest section in that regard, and... (read more)
Summary: Make predictions on all the listed predictions before the end of June 2018, and if your predictions are the best you'll win $200 after they all settle in January.
This is happening! Motivations discussed in the earlier post here, I'm running a prediction contest for the rationalist and EA communities. There are 20 predictions- seven about world politics over the next year, five about technology, six about the upcoming year for EA organisations, and two about rationalist websites, for a mix of topics with a lean towards things it would be helpful to know about. They were inspired by a variety of suggested sources- particularly Socratic Form Microblogging's world politics predictions, Slate... (read 737 more words →)
I need to take a good look over what GJO has to offer here- I'm not sure if running a challenge for score on it would meet the goals here well (in particular I think it needs to be bounded in amount of prediction it requires in order to motivate doing it, and yet not gameable by just doing easy questions, and I'd like to be able to see what the probability assignments on specific questions were), but I've not looked at it closely with this in mind. I should at least hopefully be able to crib a few questions, or more.
Sounds good. I've looked over them and I could definitely use a fair few of those.
Thanks for letting me know! I've sent them a PM, and hopefully they'll get back to me once they're free.
Summary: I think it is worth running a contest to measure ability to make accurate predictions, and am prepared to put up a prize of $200 for the winner. I need to identify a base of 20-50 propositions, all likely to settle within 2-10 months from now, and would like to ask the community for their suggestions, or for some suggested algorithms for picking them. I'd also like feedback on the idea.
I think that practicing predicting future events significantly improves calibration and provides valuable feedback on one's own level of rationality, especially around the topics you are predicting on.
More fuzzily, I think that on the community level, a greater prevalence of predictions... (read 1004 more words →)
On the positive side, I think an experiment in a more centrally managed model makes sense, and group activity that has become integrated into routine is an incredibly good commitment device for getting the activity done- the kind of social technology used in workplaces everywhere that people struggle to apply to their other projects and self-improvement efforts. Collaborative self-improvement is good; it was a big part of what I was interested in for the Accelerator Project before that became defunct.
On the skulls side, though, I think the big risk factor that comes to mind for me for any authoritarian project wasn't addressed directly. You've done a lot of review of failed projects,... (read 963 more words →)
Assuming by "it" you refer to the decision theory work, that UFAI is a threat, Many Worlds Interpretation, things they actually have endorsed in some fashion, it would be fair enough to talk about how the administrators have posted those things and described them as conclusions of the content, but it should accurately convey that that was the extent of "pushing" them. Written from a neutral point of view with the beliefs accurately represented, informing people that the community's "leaders" have posted arguments for some unusual beliefs (which readers are entitled to judge as they wish) as part of the content would be perfectly reasonable.
It would also be reasonable to talk about... (read 457 more words →)