Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: tadasdatys 11 August 2017 12:54:31PM 0 points [-]

There are way too many "shoulds" in this post. If anyone can have fun predicting important events at all, then it would probably be people in this forum. Can we make something like this happen? Would we actually want to participate? I'm not sure that I do.

Comment author: chaosmage 11 August 2017 02:50:21PM 0 points [-]

I'd definitely want to participate, and looking at the yearly predictions SSC and others do, I'm surely not the only one.

But someone would have to set it up, run it and advertise it. You don't even strictly need to write software for it. It could be done on any forum, as a thread or series of threads. It could be done here, if this place wasn't so empty nowadays.

Comment author: ChristianKl 10 August 2017 01:57:42PM 0 points [-]

I'm not sure why you consider quiz shows to be uninteresting. It's a quite successful format when it comes to gathering an audience.

Comment author: chaosmage 10 August 2017 02:38:22PM 1 point [-]

I don't know why you think that. Quiz shows need huge production values and very valuable prizes to still be interesting.

With the kind of budget that's conceivable for a startup group of amateur organizers, you have to be novel/creative to be found worth noticing outside the immediate circle of participants. Sure you could run a quiz show on a shoestring budget, but nobody is going to talk about it after.

And since this is about reaching people with ideas of thinking in probabilities and updating on evidence, everything that doesn't get talked about after is a failure. Even if the event itself was entertaining.

Comment author: ChristianKl 10 August 2017 11:46:54AM *  0 points [-]

As long as you have an existing set of questions with known answers that are unknown to participants of the game you can have instead feedback.

Public knowledge that you can find on Wikidata works if you have an offline tournament. For an online tournament, it can use data from nonpublic experiments. The CASP tournament for protein structure prediction uses that method. For our purposes, I think surveys make good experimental data.

Comment author: chaosmage 10 August 2017 12:09:20PM *  2 points [-]

But in that case, it isn't really about prediction anymore. A game like that rewards knowledge, not the ability to do research and deal with probabilistic information.

Someone who has read a lot of Wikipedia, or who happens to have read papers on topics similar to the experiment in question, could outperform someone who constructs predictions very rationally but from a different set of domain knowledge facts. This makes it closer to a quiz show, i.e. a less original and less interesting event.

A slow, online tournament (where everyone has the same internet to do research in) greatly reduces the value of blunt knowledge and makes success more dependent on the ability to weigh evidence.

Comment author: ChristianKl 10 August 2017 10:06:05AM *  4 points [-]

Currently, people who don't want to gamble for money can take part in prediction making at https://www.gjopen.com and metaculus. Your alternative suggestion is to have all the prediction making happen in a shorter time frame during an event. A special event can bring people together, but most games of Poker don't happen in the setting of a tournament.

Poker has the advantage of being able to be played casually with low feedback circles. Adrenalin rises while you play and you don't have to wait a year to see whether or not you win.

I think we need games like the credence game that have tight feedback loops. One of the core problems of the credence game is that most of the questions aren't very interesting.

I can think of a few ways to get better questions:
① Get knowledge from Wikidata and make questions with the knowledge. Test every question in the database for being a question that increases player engagement and slowly optimize your way to highly engaging questions.
② Use real world data. Sensors like heart rate monitors provide interesting data that can be predicted. Weather API provide interesting data. Time tracking software can ask me question such as whether I spent more or less than 3 hours on facebook in the last week.
③ Tie prediction to common tasks with binary outcomes. It would be possible to ask for a prediction whether or not the test suite will find an error every time I run automated tests.

Of course finding ways to actually drive economic decisions with predictions like http://lesswrong.com/lw/oe0/predictionbased_medicine_pbm/ is also important.

Comment author: chaosmage 10 August 2017 11:11:08AM 0 points [-]

A special event can bring people together, but most games of Poker don't happen in the setting of a tournament.

Oh, absolutely. But the big events are great PR and lead to lots of private games. So if there's a big event for prediction I think that helps people start to think in probabilities and conscious handling of prediction on other occasions.

Poker has the advantage of being able to be played casually with low feedback circles. Adrenalin rises while you play and you don't have to wait a year to see whether or not you win.

I think we need games like the credence game that have tight feedback loops.

That would be cool. I have a hard time imagining them though. Maybe a group of players could watch a video together, pause every couple of minutes and place percentages on a set of predictions on what happens next?

Comment author: cousin_it 10 August 2017 08:36:05AM 1 point [-]

Yes, rationality could use more competitive tests of skill. I'm not sure about having them take a year, though. Have you thought about ways to make it faster?

Comment author: chaosmage 10 August 2017 10:50:11AM 0 points [-]

A simple fix would be to include predictions about particular parts of the year. For example, have a bunch of predictions about each quarter, on top of the ones about the whole year. And then you could have an extra prize for each quarter where you score only that subset of predictions.

You could easily go more short term, like "what are your predictions for September", but I think this requires more participation and work from everyone so maybe it would be better as a second step that you do only if the relatively relaxed yearly tournament has turned out to be cool.

Comment author: chemotaxis101 10 August 2017 08:41:21AM *  1 point [-]

In order to reach a level of entertainment as near as possible to the one provided by poker tournaments you should consider building a prediction game having at least two characteristics:

1) Restricted to low-stakes events (consequences of the outcomes are minimal and affect only a small set of parties; ideally within an artificial environment rather than real world events).

2) The timeframe should be reduced to a single day.

Comment author: chaosmage 10 August 2017 09:56:48AM 0 points [-]

In that type of scenario, I think it is hard to avoid a situation where domain knowledge dominates rational handling of evidence. It might be better, I don't know. Could you describe in more detail what kind of game you're imagining?

I'm slightly biased against it because this seems even more like gambling. Specifically, like sports betting. And as soon as you involve prizes, it'll be hard to avoid being subject to gambling legislation.

Prediction should be a sport

10 chaosmage 10 August 2017 07:55AM

So, I've been thinking about prediction markets and why they aren't really catching on as much as I think they should.

My suspicion is that (beside Robin Hanson's signaling explanation, and the amount of work it takes to get to the large numbers of predictors where the quality of results becomes interesting) the basic problem of prediction markets is that they look and feel like gambling. Or at best like the stock market, which for the vast majority of people is no less distasteful.

Only a small minority of people are neither disgusted by nor terrified of gambling. Prediction markets right now are restricted to this small minority.

Poker used to have the same problem.

But over the last few decades Poker players have established that Poker is (also) a sport. They kept repeating that winning isn't purely a matter of luck, they acquired the various trappings of tournaments and leagues, they developed a culture of admiration for the most skillful players that pays in prestige rather than only money and makes it customary for everyone involved to show their names and faces. For Poker, this has worked really well. There are much more Poker players, more really smart people are deciding to get into Poker and I assume the art of game probably improved as well.

So we should consider re-framing prediction the same way.

The calibration game already does this to a degree, but sport needs competition, so results need to be comparable, so everyone needs to make predictions on the same events. You'd need something like standard cards of events that players place their predictions on.

Here's a fantasy of what it could look like.

  • Late in the year, a prediction tournament starts with the publication of a list of events in the coming year. Everybody is invited to enter the tournament (and maybe pay a small participation fee) by the end of the year, for a chance to be among the best predictors and win fame and prizes.
  • Everyone who enters plays the calibration game on the same list of events. All predictions are made public as soon as the submission period is over and the new year begins. Lots of discussion of each event's distribution of predictions.
  • Over the course of the year, events on the list happen or fail to happen. This allows for continually updated scores, a leaderboard and lots of blogging/journalistic opportunities.
  • Near the end of the year, as the leaderboard turns into a shortlist of potential winners, tension mounts. Conveniently, this is also when the next tournament starts.
  • At new year's, the winner is crowned (and I'm open to having that happen literally) at a big celebration which is also the end of the submission period for the next tournament and the revelation of what everyone is predicting for this next round. This is a big event that happens to be on a holiday, where more people have time for big events.
Of course this isn't intended to replace existing prediction markets. It is an addition to those, a fun and social thing with lots of PR potential and many opportunities to promote rationality. It should attract people to prediction who are not attracted to prediction markets. And it could be prototyped pretty cheaply, and developed further if it is as much fun as I think it would be.

In response to The Unyoga Manifesto
Comment author: chaosmage 09 August 2017 08:06:32AM 1 point [-]

I like this a lot, especially the beginning. Concise, clear and true. And the bit about the difficulty of following subtle gradients is particularly well put.

I'd like you to flesh this out more, especially because the title "manifesto" leads readers to expect more substance. Maybe add examples, or thoughts on group vs. solitary practice, or considerations about yoga for kids and yoga for seniors, and in each case make very clear what you think isn't helpful and what you think is.

Comment author: chaosmage 08 June 2017 10:32:27AM *  0 points [-]

Is there a way to get the benefit of including betting into settling arguments, without the shady associations (and possible legal ramifications) of it being gambling?

Meetup : Leipzig: Less Wrong lunch break meetup

0 chaosmage 31 May 2017 02:53PM

Discussion article for the meetup : Leipzig: Less Wrong lunch break meetup

WHEN: 28 June 2017 12:30:00PM (+0200)

WHERE: Deutsche Nationalbibliothek, Deutscher Pl. 1, 04103 Leipzig

This is a simple, friendly, informal meeting over lunch. You're welcome.

We meet in the Cafeteria of the German National Library, a stylish, cheap and relatively unknown place for decent food.

Look for a sign on a table that says "lw".

Discussion article for the meetup : Leipzig: Less Wrong lunch break meetup

View more: Next