Thank you! As I hinted in my response to Nebu above, I can see there are some personal predictions as well. I have a couple doubts about using it, though:
Overall, though, this is a great project and very relevant. I'm just being very picky.
I just made predictions, a computer program that analyzes tagged predictions, public. If you're comfortable compiling Go programs you might want to check it out. (Fair warning: the documentation and its organization hasn't quite caught up with the program.)
From a brief skim (e.g. "A Democratic candidate other than Yang to propose UBI before the second debate", "Maduro ousted before end of 2019", "Donald Trump and Xi Jinping to meet in March 2019", etc.), this seems to be focused on "non-personal" (i.e. global) events, whereas my understanding is the OP is interested in tracking predictions for personal events.
Perhaps there's some back story to this post that I missed, so forgive me if what I'm about to say has been discussed.
You might consider reading "Superforecasting: The Art and Science of Prediction," by Philip Tetlock. Or go to the Good Judgment Project web site and watch the 5-part Superforecasting master class.
First, the question has to pass the clairvoyant test. Second, you might want to have some scheme for Bayesian updating your forecast. And then you'll want to use Brier Scores (or something like them) to assess your accuracy.
If you know R, there's actually a Brier score function you can use. But I can't imagine it's very difficult to set up in Excel.
Again, sorry if I'm stating the obvious.
Thanks. This is all very relevant. And no, there is no backstory, at least not that I shared anywhere.
You might consider reading "Superforecasting: The Art and Science of Prediction," by Philip Tetlock. Or go to the Good Judgment Project web site and watch the 5-part Superforecasting master class.
Yes, I read "Superforcasting". I didn't know they had masterclass and will look it up. I suspect their teachings will be somewhat less useful for predictions of personal importance: different biases will be at play here. But it should be worthwhile to watch it anyway.
First, the question has to pass the clairvoyant test.
Actually, I think if you are going to assess your own predictions, you can afford the luxury of being a bit less specific, especially if the prediction is made in the short-term. For example, consider a made-up question:
"Will Adam be able to get back to cycling within a month [after a recent accident]?"
If Adam resumes cycling but it causes him considerable pain, I know that's not what I intended when asking the original question. On the other hand, if Adam recovers fully but starts playing rugby instead of cycling because he discovers he enjoys it more, I know the answer to the intended question is "yes". (The imprecise part of the question here is "be able to", but as long as I can reliably recall the intention when writing those words, they cause no loss of precision.)
Second, you might want to have some scheme for Bayesian updating your forecast.
Hmm. For now I was planning to make my predictions once and forget about them until the outcome is known. I'm not sure I want to spend more effort on them, at least not so early into the project.
And then you'll want to use Brier Scores (or something like them) to assess your accuracy.
If you know R, there's actually a Brier score function you can use. But I can't imagine it's very difficult to set up in Excel.
Brier score would be great at telling me how accurate I am, but not what mistakes I'm making, at least based on my very limited understanding of the metric. As a basic analysis method, I was planning to group my predictions by the forecast probability (e.g., 0%-10%, 10%-20%, ... ranges, or maybe ranges 1pp wide at the extremes and growing exponentially towards the centre, that would probably make more sense), and simply chart them grouped by tag. I must admit, my knowledge of statics has always been very poor, so I'm sure there is some better analysis/visualisation methodology.
"Will Adam be able to get back to cycling within a month [after a recent accident]?"
(Probably unnecessary word of caution) do not forecast your own behavior due to risk of reduced agency.
That's a very good point. What do you think about predicting events on which I might still have an impact? Those are some of the most important forecasts for me: I will decide whether to attempt something based on my predicted probability of success. But then my forecast might affect that probability which makes the whole thing much more complicated.
Agree! Tricky territory. I think it's fair to take an outside view as a first cut (e.g. how many people survived Everest), then very carefully evaluate if the reference class is relevant. Yudkowsky writes about this quite a bit in places cannot recall which particular place.
Thanks. I will be on the lookout for relevant writings. I'm slowly going through Yudkowsky's books/posts, so I'm sure I will stumble on it sooner or later.
Spreadsheet sounds "good enough" if you're not sure you even want to commit to doing this.
That said, I'm "mildly interested" in doing this, but I don't really have inspiration for questions I'd like to make predictions on. I'm not particularly interested in doing predictions about global events and would rather make predictions about personal events. I would like a site that lets me see other people's personal predictions (really, just their questions they're prediction an answer to -- I don't care about their actual answers), so that I can try to make the same predictions about my life. So for example, now that I've seen you've submitted a prediction for "Will my parents die this year?", I don't know what your answer is, but I can come up with my own answer to the question of whether or not *my* parents will die this year.
Spreadsheet sounds "good enough" if you're not sure you even want to commit to doing this.
Yes, exactly. I dropped the idea of writing my own software because I realised that would be overcommitting too early.
I would like a site that lets me see other people's personal predictions (really, just their questions they're prediction an answer to -- I don't care about their actual answers), so that I can try to make the same predictions about my life.
You might like to search by tag "Personal" on the PredictionBook linked below by habryka: https://www.google.com/search?sitesearch=predictionbook.com&q=personal&x=0&y=0
I do have a spreadsheet where I keep track of predictions, though only tracking the prediction, my confidence, and whether it came true or false. It's low effort and I think worth doing, but I can't confidently say that it has improved my calibration.
I've been thinking how I can improve my accuracy predicting events of personal interest (e.g., "Will my landlord get the washing machine fixed within the next two weeks", or "Will my parent die this year" for a more extreme example). Betting markets will not help me with that.
At first I thought about creating dedicated software that gathers such predictions, the final outcomes of predicted events, and presents their accuracy so that the user can spot bias. Then I realised a simple spreadsheet might suffice to gather data at first and assess how useful this is. And if the need arises in the future, it should be easy to import into dedicated software, provided that all the relevant data is already there.
Does anyone track their personal predictions? If so, what methodology do you use, and did it allow you to improve your accuracy?
As an RFC, here's the spreadsheet layout I have on mind: