= 27d567bc2d48d9f40e9ccc20ea370b15)
Morendil comments on [LINK] Get paid to train your rationality - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (55)
Apparently the only way to know is to try. It seems likely that there is such a restriction. I'd estimate a better than 70% chance that I get turned down. :)
I got an email an hour ago from the study saying I was accepted and taking me to the initial survey (a long one, covering calibration on geopolitics, finance, and religion; personality surveys with a lot of fox/hedgehog questions; basic probability; a critical thinking test, the CRT; and then what looked like a full matrix IQ test). The message at the end of all the questions:
So I'm marking me as accepted, anyway.
And the "tournament" is now begun. Just got email with login instructions.
Looks somewhat similar to PredictionBook, actually. :)
I did all my predictions last night immediately after the email showed up, so that meant I got to place a lot of bets at 50/50 odds :)
(Then I recorded everything privately in PredictionBook. No point in leaving my predictions trapped on their site.)
Interface-wise, I don't like it at all. I'm still not sure what exactly I am betting at or with, compared to PB with straight probabilities or Intrade with share prices.
Did you take the "training refresher"? That includes a general-knowledge test at the end which scores you on both calibration and resolution. My results were pretty poor (but not abysmal):
I'd be curious to compare with yours if you'd care to share.
Without actually going through the whole refresher, it seems to be the same; when I did the training, I don't remember that calibration/resolution test. Perhaps that is one of the experimental differences.
I didn't remember that test from earlier, either. Worth checking out? I don't mind accidentally unblinding a little if it is an experimental/control difference - curious folks will be curious.
I just went through the whole thing again; there was no test of that kind at the end. (What there was was the previous multiple-choice quiz about some example forecasts and how they went wrong.) Looks like this is an experimental/control difference. I'd rather not discuss that bit further - this isn't about possibly life-or-death drugs, after all, and I already know where I can find calibration tests like that.
Fine with me. :)
BTW, look what I found. Did you know about this one?
Looks like someone is being very naughty. I've asked him on Twitter.
Have you entered any comments on your predictions at the GJ site? (You're supposed to enter a minimum number of comments over one year, and also a minimum number of responses to others' comments. My understanding is that this will in time be run as a team game, with team play conventions.)
From my first experiences, I'm assuming the scoring will be pretty much as with PB.com - based on probability. Their model seems to be calibration/resolution rather than the visual "slope" representation.
Comments? I don't see any relevant fields for that, checking right now, nor does my 'About' include the substring "comment". Another experimental difference, I guess...
The "Why did you answer the way you did" field. I've been assuming we're both using the same underlying app, i.e. Crowdcast. But perhaps we're not...
I'm in; pleasantly surprised.
This bit from the final registration page is interesting - "But one other requirement for forecasters has changed. We can welcome those who are not US citizens." Implying that at some prior point non-US citizens were not accepted.
That is awesome!
Especially (mischievous mode ON) as I've only implied, not outright stated, that I've applied.
Mischievous mode OFF - that's a problem in arbitrating predictions, btw - the potential for ambiguity inherent in all human languages. If I hadn't in fact applied (I have), how should the prediction that I am "turned down" be judged?
I should use PredictionBook more often but I don't, partly due this kind of thing, also due to the trivial-inconvenience effort of having to come up with my own predictions to assess and the general uselessness for that purpose of the stream of other users' predictions.
Other than Tricycle folks, is anyone here on LW officially (or unofficially) "in charge" of maintaining and enhancing PredictionBook?
I have some sort of moderator power; I am de facto in charge of the content house-keeping - editing bad due-dates, making private bad or long-overdue-unjudged predictions, criticizing predictions, etc. I also make and register hundreds of predictions, obviously.
(In addition, I have commit access to the codebase on GitHub, but I don't know Ruby, so I will probably never make use of said commit-bit.)
One thing that would probably greatly improve PB for my purposes is a tagging / filtering system, so that you could for instance pick out predictions about consumer devices or predictions about politics; or conversely leave out some uninteresting categories (e.g. predictions about the private lives of particular PB users, which I interpret as pure noise).
Google is not sufficient, I take it?
No; I just tried the query "consumer electronics site:predictionbook.com", and that only returned 1 hit; I know there are more (including one I just made and another I just voted on). It really is the lack of user-supplied meta-information that prevents useful querying, not the lack of a UI for doing so. The UI encourages predictions to be written very tersely, and doesn't supply an extended-info field when you make a prediction.
PB.com is quite possibly the least well executed idea out there that I keep not giving up on. :)
Ah, that's what you meant by tags. Yes, that would be nice. On the other hand, I rather doubt that tags would instantly create massive demand for PB's services - other places like Intrade have well-categorized predictions/bets, and none of them have seen traffic explode the moment they implemented that feature.
If you really found tags all that valuable, you could start doing them inside comments. Go over the 969 upcoming predictions and add comments like 'tags: personal, exercise' or 'tags: America, politics'. Later, it'd be even easier to turn them into some real software-supported tags/categories, and in the meantime, you can query using Google. This wouldn't even take very long - at 30 predictions a day, which ought to take 10 minutes max, you'd be done in a month.
(I doubt you will adopt my suggestion and tag even 500 predictions (10%). This seems to be common to suggestions for PB: 'I'd use and really find PB useful if only it were executed better in this way', which of course never happens. It's starting to remind me of cryonics.)
Preliminary report: this isn't going to work, not without drastic contortions in the choice of tags (which IMO kills the effectiveness of the tactic). For instance, from my first set of 30 I tagged a number with the tag "personal", predictions which only concern one user (or two acquainted with each other) and that I don't want to see because I can't effectively assess them. The Google query including "personal" returns close to 30 spurious results: for instance those containing "personal computer" or "personal transportation". (A temporary workaround is to include the term "tags" in the query, but this will cease to work once a greater fraction of predictions have been tagged.)
You are correct about the likely outcome, but I think I've just proven your model of the underlying reasons wrong: I won't do it because it won't work, not because I lack the conscientiousness to do so, or because I'm too selfish to take on an effort that will benefit all users.
How many times was a new feature implemented as a test of such a hypothesis?
PB.com seems like it would be a great place for things like A/B testing and other tactics in the "Lean startup" repertoire, but what actually seems to be the case is that the site isn't under active development any more; no one is apparently trying to develop traffic or usage by improving the user experience there. (This isn't to slight your own efforts or your obvious enthusiasm; merely my best current hypothesis.)
(I'm finding the comparison with cryonics ironically apt, as a fence-straddling LW user who's not giving up on the entire project despite facing, as a non-US citizen, a battery of obstacles that I suspect also apply in the US, where they're just less obvious and as a result people take it for granted that things will "just work". Though it's more likely that the comparison is entirely besides the point and just a red herring for the purposes of present discussion.)
I'll try that, for a minimum of 30 predictions.
Hmm, I think this is a good idea. When I make a prediction or comment on it I will add tag remarks. It is non-ideal hack but should help a little bit.