Big news: We massively overhauled Metaculus’s code to add new features, improve others, and make it easier for open-source contributors to iterate with us.

If that’s you, and you want to help shape the future of the platform, click here for the Metaculus code repo and to learn more.

There you'll find instructions for setting up a local development environment, and instructions on creating an issue. We also curated a list of issues suitable for newcomers. It's a great place to start.

So join us: We'd love to send you a Metaculus hoodie for your efforts.

What’s new? What’s improved?

The highlights

Profiles

User profiles are now easier to navigate and more informative, with an Overview tab featuring key stats, medals, and a calibration curve.

Notifications

They’re simpler to set up: Just click ‘Follow’ on a question, and we’ll notify you of new comments and changes in the Community Prediction. Want more control? You can customize which notifications you receive.

Comments

A new WYSIWYG (What You See Is What You Get) editor provides seamless text styling and embedding of images and forecast questions. Private Note filtering helps you locate any you’ve written, and nested comments make it that much easier to follow the conversation.

Question Feed

The main feed sports a simplified interface and is split in three:

• Feed Home — all questions
• My Predictions — questions you’ve made a prediction on
• My Questions and Posts — questions and notebooks you’ve written

API

Do you use the Metaculus API or compete in the AI Forecasting Benchmark Series? We simplified some of our GET responses. You can find breaking changes to the Question endpoint and suggestions for how to update your flow here.

Zoom Timelines

You can zoom in on timeline graphs to focus on the last two months, week, or day. You can also zoom in on embedded questions and even set the default zoom.

Those are the highlights. You can continue reading for a comprehensive list of other updates we’ve made. We’d also welcome any feedback or questions you’d like to share in the comments section.

Additional Updates

Track Records

Calibration is now front-and-center on our simplified track record pages—for both the Metaculus platform and for individual forecasters. We have big plans for making it easier to understand and share forecasting performance—but for the near term we’re glad to make track record pages more approachable.

We've also changed how calibration is presented. Before, we compared a 50% Jeffreys confidence interval around the forecaster’s calibration to a (sometimes unattainable) “perfect calibration”. Now we compare the forecaster’s actual calibration to the 90% CI of a (hypothetical) perfectly calibrated forecaster. We feel this is simpler and more straightforward.

Settings

Options on your Settings page are more clearly laid out. Hiding the Community Prediction will now hide it on every Open question on Metaculus. If you want to see an individual CP, a new button on question pages will temporarily reveal it.

Divergence

We measure the difference between two predictions (for the divergence sort and for Community Prediction change notifications) using the Jeffreys divergence. It correctly treats a 1%→2% change as larger than a 50%→51% change (~18 times larger in fact).

Notebooks & Discussion Posts

We merged these into a single content type, with a WYSIWYG editor that simplifies embedding questions and images.

Question Tile Fields

Feed question tiles had become overcrowded, so we removed some of their fields. If there’s information you’d like back, let us know in the comments.

Question Writing

We streamlined question writing by removing some unnecessary fields, and we gave the workflow a more appealing look.

Community Prediction

We only show a single aggregate forecast on question pages: the Community Prediction. This is a time-weighted median for binary and multiple choice questions, and an average for continuous questions.

These are the aggregates we were already showing on questions by default. Now, in order to view other aggregates, like track record-weighted Metaculus Prediction, or the non-time-weighted median or average, you’ll visit the new 'Aggregation Explorer' in the 'More' navigation menu above. The change makes it easier for us to update and introduce new aggregates. We look forward to sharing more on that soon.

Continuous Distribution Minimums

Continuous probability density functions (PDFs) now correctly display a minimum of 0.01 (with 0.1% outside each open bound). This was already the case for the predictions Metaculus stores and scores, but previously the interface didn’t display this information properly.

Permissions

The new simplified permissions system will be invisible to most users, but it will make admins’ lives easier, which will be a boon to question development and moderation in general.

Language Selector

We've introduced a language selector, which allows you to switch the "static" text (e.g., buttons, descriptions, various pieces of copy) to Czech, Spanish, or Chinese. Please note that this only applies to static content for now, and to change "dynamic" text (e.g., question descriptions, comments, etc.) you will for now have to rely on your browser’s translation tools. We think it's important to have manual translation of technical terms in particular to ensure clarity and accuracy. Looking ahead, we plan to significantly expand our language options on Metaculus.

Old site

In case you miss it, or want to check how something used to look, the old site is available at old.metaculus.com. We will be keeping it online for a while for convenience, but note that anything you do there starting now will never be migrated to the new site.

So take a look around at the rewritten Metaculus, and let us know what you think in the comments below!