Minor bug/sadness report: I did the first question, then it prompted me to log in before revealing the answer, and AFAICT the question-answer didn't get recorded and now I can't find that question again and am curious and sad.
Thanks for the feedback! We've fixed the issue and you should be able to see our default question again after login.
I've used this a bit, but not loads. I prefer fatebook, metaculus and manifold and betting. I don't quite know why I don't use it more, here are some guesses.
- I found the tool kind of hard to use
- It was hard to search for the kind of information that I use to forecast
- Often I would generate priors based on my current state, but those were wrong in strange ways (I knew something happened but after the deadline)
- It wasn't clear that it was helping me to get better versus doing lots of forecasting on other platforms.
This is great. It seems to directly hit at making me more "genre-savvy". If I were spending lots of time forecasting, I would probably do this for an hour a day.
The only improvement I can think of is to have forecasters study in hindsight what their ex-ante probability should have been, and show you that aggregate number too. But that seems like a lot of work plus potentially biased.
I cannot claim to have used this tool very much since it was announced, but this sure seems like the way to go if one wanted to quickly get better at any kind of medium or long-term forecasting. I would also really love a review from someone who has actually used it a bunch, or a self-review by the people who built it.
I really like this idea and I've been enjoying using the site!
I have noticed, however, that some of the questions include how they were resolved in the description. This is not a huge issue given that the system depends on self-regulation and there is already a mechanism for skipping questions for which users have prior knowledge. That said, it was confusing when I saw the question below because I thought "if the resolution of the forecast is embedded in the description, it would be silly to include it in the question set. Therefore it must not actually be giving me information about how the question resolved."
This only happened for 2 out of the 30 questions I looked at, but it seems like something that people should be aware of.
What's to stop someone from pretending not to know the answer to a question they actually do know the answer to so they can miraculously "predict" it better and get more points?
Unfortunately, there’s no fully general solution to prevent cheating. We rely on people’s honor to skip questions they already know and intend it to be used mainly as a tool for training and competing against people you trust.
TL;DR: Visit pastcasting.com to forecast on already resolved questions you don’t have prior knowledge about to get quick feedback on how you’re doing.
We may try to coordinate multiplayer sessions in our Discord
Motivation
We want to make it easy for people to get better at forecasting. Existing tools are not well optimized for practicing relevant skills.
Pastcasting
With pastcasting, you can:
How it works
Question sources
Our questions are currently pulled from resolved Metaculus and GJOpen questions.
Scoring
We use relative log scoring against the original crowd forecast at that time. This means that you will receive zero points if you submit the same value as the crowd. The scoring rule is also strictly proper, meaning that your expected score is maximized if you report your true beliefs.
Vantage Search
Our preliminary results (in yellow) come from a search api with a restricted date range up to the vantage date. To further reduce information leakage (from the website changing its contents), we then pass the results through the internet archive api (in blue). This tends to be much slower, so some pastcasters opt to use the preliminary results directly.
Prior Knowledge
In many cases, users will have specific knowledge of how a particular question was resolved (from reading about it in the news, participating in forecasting that question, etc). We provide a button to skip these questions, and over time we will show questions that many users know the answer to less often.
Give it a try!
If this sounds useful, give it a try and give us feedback on what features would be the most useful and any aspects of the site you find confusing.
Such as Metaculus, Manifold Markets, etc…
Such as https://www.openphilanthropy.org/calibration, https://acritch.com/credence-game/, Wits & Wagers.