Epistemic status: I'm not the first person to think of this, and I'm not sure if this is possible. But I want to flesh out some options for consideration and get feedback. Sorry if it's long.

WANTED:

A crowd-sourced site that reliably presents important pros and cons on any/all topics and hopefully leads us collectively closer to consensus.

This has been on my mind for a few years now. Figuring out truth in the digital age is often tricky and sometimes downright impossible. (Is minimum wage a good thing? Should I adopt a paleo or keto or vegan or Shangri-la diet? What do we really know and not know about the historical Jesus?) Especially when it comes to political discussion, answers often depend on more details than an average person has time or energy to research—and even if they did, they may still leave out considerations or solutions that someone else may think of.

Some links where other rationalists think semi-related things: Robin Hanson wants AI to help us towards consensus because Wikipedia isn't up for the job. Scott Alexander promoted adversarial collaborations to see if they could reach consensus. (I swear, I saw more examples of people wishing for something like a debate website, but I wasn't writing them down and searching for them has failed.)

THE WISHFUL THINKING PART

Now let me be more specific about what my imagined website (hereafter, Site) needs to do.

On matters of truth, it needs to support epistemic arguments for why we should believe or not believe particular claims. On matters of action, it needs to provide important pro/cons of taking that action. Site must have a method of allowing the best arguments to rise to the top. That's the most basic functionality.

Additional Feature A: Because the Site is crowd-sourced, anyone can propose possible actions and possible modifications to actions. In theory, this could mean that Site could be a platform where a crowd-written law could have its details hammered out by many affected parties. Creative solutions could have a chance to be reviewed and, if well-thought-out, could rise higher in the public consciousness.

Optional Feature B: I really, really would like it if, within the structure of arguments (for either epistemic-truth topics or proposed-actions), Site distinguished between truth-claims and value-claims. This might be especially unrealistic, since no one naturally lays out arguments that way. But gosh dang it, I really want it to.

Optional Feature C: A number of rationalist posts have discussed how useful it would be for society to have a way to coordinate on certain problems: specifically, having a conditional "I will adopt X if Y% of other relevant people also do". It might be possible to set up Site so that proposed actions that need a minimum number of people to adopt it in order for it to be worthwhile offer the option of conditionally committing to adopt it and sending out notifications when thresholds are reached.

THE NITTY-GRITTY

Obviously, this is a LOT to ask from a website. Before I go into everything that could go wrong and all the reasons it might not even be possible, let me lay out a couple of operational options.

Contrast Voting

Reddit and other up/down-vote systems suffer from certain effects, an important one being that late-to-the-game comments don't stand a chance against early popular comments, even if the later ones are higher quality. To combat this, Site should not allow up/down votes on individual items/comments/whatever. Instead, Site should present two items and ask for a vote on which one is better. In this way, Site can give better rankings to items. A new argument, perhaps one that presents a new, more recent study finding, can quickly outrank the previous top argument if, when those two arguments are presented, most people vote that the new one is better.

Contrast Topics

The Contrast Topics option would not have a page titled "Is God real?" nor a page for "We should outlaw chicken cage farming." Instead, topics would be presented in terms of possible alternatives. "The Abrahamic God is real" could be separately contrasted with "There is no God" and, on another page, contrasted with "There is a divine energy". "We should outlaw chicken cage farming" could be contrasted with "We should eat less chicken and let the market adjust chicken cage farming downward" or "We should slowly phase out chicken cage farming according to this detailed plan I've written out" or "We should wait until chicken-like lab meat is available, and then outlaw chicken cage farming".

Visually, this might be tricky, but a "Chicken cage farming" page might list all of the proposed actions related to it, and a user could select two actions and click "next" to see the pros and cons of those two actions relative to each other. (These would be more like pros of A vs pros of B, not pro/cons of each; a con of B would be listed as a pro of A and vice versa.) These headings might be treated as tags/labels rather than a strict hierarchy of categories, so that a possible action or topic might appear under multiple headings if it's relevant to all of them. Headings might, in turn, have both epistemic truth-claims and potential actions listed under them.

Open Forum

In this option, individual users compose complete arguments, either pro or con. They are allowed and even encouraged to steal what others have written, add/subtract/modify it as they like, and submit it as a new argument. Contrast Voting is used within "pro" and within "con" to rank these arguments and present the highest ranked at the top. Probably only the top one or two submissions will show by default (the rest reached by an expandable + or "show more" or such), since submissions are expected to (eventually) aggregate all the best arguments.

Wikibate

This is an alternative to Open Forum. It works more like Wikipedia; everyone is allowed to directly edit one single page. A pro and con side are provided, and we see if a version comes out that makes enough people happy enough to be stable. A behind-the-scenes contrast voting option might be used to decide which order the possible arguments appear in, but the phrasing of the arguments would have to satisfy people or be edited by someone.

Personal Expertise Stories

In normal life, people rely heavily on personal experiences when forming their beliefs. If Site doesn't sometimes take this into account, it will feel shallow to many people. Argue the facts of abortion all you want, but if you can't include someone's vivid description of having a loved one die from a back-alley abortion or another person's horror at seeing what they were told is a "lump of tissue" that turns out to look exactly like a baby, you're missing important context.

It is also the case that sometimes hearing expert testimony is helpful. Lawmakers often rely on it. Experts don't always agree, but a claim from an expert can and should carry a different weight than the same claim from a non-expert.

On a crowd-sourced site, anyone can claim they're an expert or make up b.s. stories about what happened to them. A possible solution is to allow people to submit a "personal expertise/story". If submitting one, you have to include personal contact info like a phone number. Some number, say 10, people can apply to verify a story. Only approved verifiers are given access to the story submitter's contact info. The verifiers promise to come back and report on the status of their verification and not to abuse the contact info. (A complaint from a story submitter results in that verifier being unable to do any further verifications, and possibly banned from the Site.) The verifier calls the person and talk to them, get details if possible, like other people or companies they can talk to, to verify parts of the story or claim. In the end they report back, maybe on a 5-point scale, whether they believe the person's claim, and the net verification result from all the verifiers is displayed with the story. (E.g. a green bar showing 3.5/5, with a (2) next to it indicating only 2 verifiers have submitted responses.)

Consequences Before Arguments

For proposed actions, before the actual pro/con arguments appear, I would like to see a list of possible consequences that might result from taking that action. While I don't want Site to adopt consequentialism as an official philosophy, I remember (Google, you fail me) an article that claimed that people demonstrated less partisanship when they were asked to think about the consequences of possible laws. A potential consequence is itself a truth-claim ("If we do X, then Y will happen"), so that should link to its own page.

Login Voting

Generally, I want people not to have to create accounts in order to participate on the website. IP addresses can be recorded, like Wikipedia does, for submitting/modifying arguments, proposing actions, and creating topics. That should minimize the barrier to entry and encourage participation. However, all voting should require logging in, to minimize one person voting multiple times, and to accurately track if someone changes their relative rankings of two arguments.

Parallel Axes Voting

There is no one standard for what makes an argument "better" than another. Relevant measurements include: accuracy, importance/applicability, thoroughness, kindness, formatting/grammar/spelling. One option might be to allow users to Contrast Vote two arguments in multiple categories separately: Argument A has more important points than B, but B has better grammar than A, and they are the same on kindness. Parallel Axes Voting is more relevant if Site uses the Open Forum style, and not so much if it uses Wikibate.

THE PROBLEMS, OH SO MANY PROBLEMS

• I don't know how to make this website. I know a little html and css, and I have a vague idea that Amazon Web Services could be initially used to host the site. But I know nothing about how to use AWS, how to make a database and a website talk to each other, how to have user accounts and secure logins, etc. I'd either need a lot of help or someone else would have to do it entirely or I'd need to invest a ton of time into basically learning a new profession to make something that might fail.
• The most popular arguments aren't necessarily the best.
• People might sacrifice truth for simplicity. Given two arguments that make the same point, one that presents the point more simply and understandably is better than one that doesn't. However, users will sometimes be offered the choice between a simple, understandable argument that isn't accurate and a difficult one that is more correct. I don't know of a way to discourage upvoting the simpler one over the more accurate one. Worse, if all the arguments on the inaccurate side are simple and easy to understand, and all the arguments on the more correct side are difficult to understand, Site could backfire and have the effect of convincing people of something that isn't true.
• Some topics require a whole background course to understand. Israel/Palestine, the mortgage crisis...there's some things that require so much background knowledge (even to be familiar with the relevant terms) that I'm not sure it's possible to create arguments that the average reader could understand without also presenting some sort of background course on the topic. [Maybe this could be ameliorated by limiting the voting on certain topics to people who have accessed and agreed that they have read a background page that verified experts have approved?]
• I don't know if Site will actually promote consensus. I think seeing your opponent's best arguments instead of only their worst will reduce partisanship a little. I think that comparing multiple possible actions on a topic instead of only one or two will help some. But that might not be enough.
• Having all arguments divided into either pro or con instead of one single narrative might not shift people into a consensus. Relatedly, it might be difficult for the two sides to adequately interact with each other and respond to each other's views when everything is listed on one side or the other.
• Seeing the pros and cons of both sides might actually make untrue beliefs (flat Earth, for example) seem more legitimate or reasonable than they are; the epistemic imbalances might not come across effectively.
• Much disagreement includes trusting sources differently. If an argument about whether something did/would happen or not boils down to "Breitbart/Slate said so", Site might not have a way to resolve that.
• A website might be innately insufficient; personally caring about another person you know might be required for consensus.
• A sub-point on this one is that in the Wikibate style, I don't know if arguments can reach a reasonably stable state. Wikipedia does sometimes have those background pages for arguing about whether an article should say one thing or another; Wikibate might try those for settling disputes, deciding whether subtle differences are similar enough to count as the same thing or not, and such. But contentious things are contentious, and that might not be enough.
• Some algorithms and UI details need to be worked out.
• How can the conditional-cooperation option be implemented? If different people have different thresholds that need to be met before they will do it, how does Site take that into account? How does it handle identifying the subpopulation of the whole planet that would need to conditionally commit to certain measures? (E.g. all corporations will follow some environmental rule if all the others do too—the CEOs or other higher-ups in the corporations would be the only relevant population that would need to meet the threshold, not every individual on the planet).
• What formula should Site use for taking user's Contrast Votes and turning them into an overall rank? What happens when people be their inconsistent selves and like A better than B, B better than C, but C better than A?
• How do we get users to rank new submissions? Do we list new ones separately above the already-ranked ones? Force a pop-up that asks people to rank a newer submission with a random older one before they continue reading that page? Offer a sidebar that says "Here's some new arguments, please review and rank them:"?
• Numbers. This is especially relevant when listing possible consequences of actions, but sometimes Site needs to handle numbers delicately. "One more person each year will die if we do this" is different from "100,000 more people will die each year if we do this". Listing all the possible estimates that individuals pull out of their...err, hat would be overwhelming to look at and consider. Do we create a separate page for arguments over what the number will be and institute Contrast Voting on the results so that the original page shows the most popular estimate? List upper and lower bounds: "Between 1 and 100,000 people will die"?
• How does Site handle nesting? How far down do we let nesting continue before deciding that two options are too similar to bother listing them separately? Does Site make you compare only the lowest-nested level with each other, or you compare higher nested levels too? (That is, is there a page for Abrahamic God vs. No God and also a page for prayer-answering Baptist God vs. No God, even though prayer-answering Baptist God is nested under Abrahamic God? Or can you only compare prayer-answering Baptist God to non-so-interventionist Baptist God? If so, how do you get agreement on what aspects of God all the Abrahamics agree on?)
• Is a summary possible? Would it be possible for a page to present a sort of consensus summary? (Something like "People prefer immediate outlawing of chicken cage farming if they value reducing the total amount of suffering of living creatures more than they value reducing human suffering alone, and people prefer gradual phasing out of chicken cage farming if they value reducing human suffering alone more than they value reducing the total amount of suffering of living creatures".)
• How does Site balance covering all the options with presenting a number of possibilities that people will actually read? Sometimes it will be enough to list all the options, put the most important/relevant ones at the top, and let people read as far down as they want. Sometimes that might not be enough. For instance, if you want to list the consequences of a possible action before you list the pro/con arguments for that action, then what do you do if there's 50 consequences proposed? Cut it off at an arbitrary number (say, only show the 5 most important)? Display any that, say, 50% of people vote should be visible without being hidden behind a "see more" option?
• How is moderation handled? Like, I have no clue. This probably varies depending on whether Site uses Open Forum or Wikibate.
• There's a lot more details to work out. I think I have some implicit imagery in my head as to how Site looks and operates that I might not have laid out explicitly here, but a lot of it has yet to be worked out. And doubtless huge new problems will be encountered once actually trying to make the site.
• Getting Site to be popular enough to be useful is difficult. I definitely don't have the social network or following to pull something like that off. I don't really know anyone who does. The best I could do is email a link (once it's set up) to random famous people and say "Please check this out". Maybe if someone writes up a "6 Reasons to Check Out This Website (And Laugh)" article and submits it to Buzzfeed.
• Funding. If it does become popular, decisions will need to be made about advertising and how to pay for server maintenance and whatnot.
• Site needs a cool name. All I've come up with is Wikibate (if it's set up that way) or BetterThink.

In the end, despite the problems, I think Site would be more worth having than not.

So what do you think? Is Site even possible? Is there a better setup than Open Forum or Wikibate? If not, which of those two is better? Should the Personal Expertise Stories option be included? What other details could Site implement to be successful? Are any of you willing to work on it? Is it worth my time to work on it?