How can I remove an estimate I created with an accidental click? (Said accidental click is easy to make on mobile, especially because the way reactions work there has habituated me to tapping to reveal hidden information and not expecting doing so to perform an action.)
This doesn't work. (Recording is Linux Firefox; same thing happens in Android Chrome.)
An error is logged when I click a second time (and not when I click on a different probability):
[GraphQL error]: Message: null value in column "prediction" of relation "ElicitQuestionPredictions" violates not-null constraint, Location: line 2, col 3, Path: MakeElicitPrediction instrument.ts:129:35
1
1Sorry about that, a fix is in progress. Unmaking a prediction will no longer crash. The UI will incorrectly display the cancelled prediction in the leftmost bucket; that will be fixed in a few minutes without you needing to re-do any predictions.
1FWIW I would prefer to have to click to show the distribution, so I could vote before being anchored.
You can change this in your user settings! It's in the Site Customization section; it's labelled "Hide other users' Elicit predictions until I have predicted myself". (Our Claims feature is no longer linked to Elicit, but this setting carries over from back when it was.)
2Bug report: It does have the amusing property that, if you hover over a part of the claim where others have left votes, the text underneath vanishes. Normally it would be replaced with the names of the users who voted, but now it shows no text. This doesn't reveal the key identity bits, but does reveal non-zero bits about what others think.
I'm somewhat surprised to see the distribution of predictions for 75% on FrontierMath. Does anyone want to bet money on this, at say, 2:1 odds (my two dollars that this won't happen against your one that it will)?
(Edit: I guess the wording doesn’t exclude something like AlphaProof, which I wasn’t considering. I think I might bet 1:1 odds if systems targeted at math are included, as opposed to general purpose models?)
oh, I like this feature a lot!
what's the plan for how scoring / checking resolved predictions will work?
I don't think I would want any kind of site-wide scoring, since I explicitly don't want people to have to worry too much about operationalization.
I do think if people keep using the feature, we'll add some way to resolve things, and resolved markets should have some way of showing whose predictions were right, and send people notifications. And maybe we'll also show people's predictions on their profiles at some point.
If the thing really takes off, of course there is a whole world of integrating prediction things more deeply into LW, but that seems somewhat unlikely given Manifold's existence and success.
1We don't have any plans yet; we might circle back in a year and build a leaderboard, or we might not. (It's also possible for third-parties to do that with our API). If we do anything like that, I promise the scoring will be incentive-compatible.
. . . Okay, I'll bite.
Edit: And-
Kinda. There's source code here and you can poke around the API in graphiql. (We don't promise not to change things without warning.) When you get the HTML content of a post/comment it will contain elements that look like <div data-elicit-id="tYHTHHcAdR4W4XzHC">Prediction</div> (the attribute name is a holdover from when we had an offsite integration with Elicit). For example, your prediction "Somebody (possibly Screwtape) builds an integration between Fatebook.io and the LessWrong prediction UI by the end of July 2025" has ID tYHTHHcAdR4W4XzHC. A graphql query to get the results:
query GetPrediction {
ElicitBlockData(questionId:"tYHTHHcAdR4W4XzHC") {
_id predictions {
createdAt
creator { displayName }
}
}
}One prediction I'm interested in that's related to o3 is how long until an AI achieves superhuman ELO on Codeforces.
OpenAI claims that o3 achieved a Codeforces ELO of 2727 which is 99.9th percentile but the best human competitor in the world right now has an ELO of 3985. If an AI could achieve an ELO of 4000 or more, an AI would then be the best entity in the world at competitive programming and that would be the "AlphaGo" moment for the field.
An AI system replicating itself seems very unlikely because AI labs are hopefully and presumably protecting against that in particular. However, there are many other dangerous things an AI system could do that aren't self-replication and are often worse. It also seems that if that does happen, we are doomed since AI labs are trying their hardest to prevent that, and if they fail to prevent that, we have a self-replicating non-aligned AI, and so we are screwed.
2024 is drawing to a close, which means it's an opportune time to make predictions about 2025. It's also a great time to put probabilities on those predictions, so we can later prove our calibration (or lack thereof).
We just shipped a LessWrong feature to make this easy. Simply highlight a sentence in your comment, and click the crystal-ball icon on the toolbar to turn it a prediction that everyone (who's logged in) can put probability estimates on. The result will look like this:
Some more probabilities that seem cool to elicit (basically all about AI, because that's what's on my mind, but it would be great to have some less AI focused predictions from others)[1]:
Unless otherwise specified assume all predictions are about the state of the world at midnight PT, Dec 31st 2025. Also some things won't be perfectly operationalized. Assume that I am going to be judging the resolution using my best judgement.