Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

PeerInfinity comments on Debate tools: an experience report - Less Wrong

38 Post author: Morendil 05 February 2010 02:47PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (72)

You are viewing a single comment's thread.

Comment author: PeerInfinity 07 February 2010 05:22:56AM *  6 points [-]

I would recommend that we try to create our own debate-mapping tool. It might end up being surprisingly easy.

I've already used PHP, GraphViz, and MediaWiki to implement a vaguely similar project, the Transhumanist Wiki Scenarios Map.

Unfortunately, that project ended up being less useful than I had hoped, and has been abandoned for now.

Today, I made a rough sketch of what a debate-mapping tool based on these tools might look like.

A VERY rough sketch.

Pretty much every detail is probably going to need to be changed in order for it to be useable.

Anyway, here's a link to that experiment

Once again, It didn't turn out as well as I had hoped.

The basic idea is that you take a chat log of a debate, and add some annotations, marking which are the main claims of the argument, and indicating which arguments support or oppose which other arguments.

Then, run a script on this annotated chat log, and it will output a graph of the arguments in the debate.

One advantage of this method is that the text and the annotations can be updated as the debate continues, and the graph will be updated to match this new data.

Some ideas for things to change:

change the formatting of the annotations. the word "claim" is unnecessary

set up the actual PHP script. These example graphs were generated by manually formatting the annotations in the graphviz format.

set up different formats for the output. A graph is not the most useful format. A better idea would be a table summarizing the info for each of the claims:

  • the text of the claim
  • a list of supporting claims (preferably with clickable links)
  • a list of opposing claims
  • a list of claims that this claim supports
  • a list of claims that this claim opposes
  • a list of who agrees with this claim
  • the results of any of the other calculations that were performed on this graph. Maybe something with probabilities?
  • more?

perhaps each claim could have its own wiki page, similar to how the scenarios map works

add more keywords, besides just "supports" and "opposes". Some examples are:

  • relies on
  • is an example of
  • is a counterexample to
  • assumption
  • more?

add a way to indicate which speaker agrees with which claims, and deduce from that which conclusions are supported by implications of their assumptions

set up the script to automatically generate the graphs as the wiki page is updated

more?

Comment author: Morendil 09 February 2010 02:23:41PM *  1 point [-]

Yes, dissecting a conversation like that is the kind of thing I had in mind.

A further intent, and I agree with Eliezer here, is to capture and provisonally settle arguments as they occur, to avoid discussions that go around in circles. So it's not about just mapping one conversation, but also about being able to link further conversations back to that formal map.

By "provisionally settle" I mean something like agreeing that whether you should spend your money on cryonics is an expected utility trade-off. If you think there is an ethical issue such that even wanting cryonics to work is bad, or sinful or what have you, there's no point discussing the price. The discussion has a tree structure, as suggested in this thread by clockbackwards and Jennifer.

Possibly you don't need much more of a tool than a good outliner. Perhaps not even that: if you set out to write the "definitive" article on the best known arguments for and against cryonics, just setting out everything in one place, that could be enough. I very much like the "roadmap" format, as in the whole-brain emulation roadmap (pdf link).

The written word is hard to beat.

I do appreciate the work you've done on these Wiki pages, btw. Perhaps, as Vladimir Nesov commented somewhere, they won't go much further; but it's also good that they exist and collect in one place information that has taken some effort to gather.

Comment author: PeerInfinity 09 February 2010 06:24:18PM *  7 points [-]

The idea I keep coming back to is something that works basically the same as how the Transhumanist Wiki's Scenarios Map currently works, but expanded to include a few more features.

The way it is set up now:

  • Each claim in the argument would have its own node in the graph.
  • Each connection between claims would have its own node in the graph.
  • Each node in the graph would have its own wiki page.
  • Each wiki page contains all of the information about the node, including tags that specify how it connects to the other nodes.
  • Nodes for claims would include a description of the claim, any debate about the claim, links to supporting, opposing, supported, and opposed claims, and any other relevant information.
  • Nodes for connections between claims would include an explanation of why the claims are connected, debate about whether the connection is valid, and any other relevant information.
  • The graph is automatically generated by a PHP script that scans all of the wiki pages, and parses the tags.
  • Clicking on any node in the graph loads that node's wiki page, which shows a graph of nearby nodes.

A BETA VERSION OF THIS HAS ALREADY BEEN UP AND RUNNING FOR MONTHS!

But noone has shown any interest in this at all.

And I don't know why noone is showing any interest.

  • Is it because the core idea of the system is so unworkable that it would be best to just abandon the whole project?
  • Is it because noone has any use at all for any system even remotely like this?
  • Is it because noone has heard about it?
  • Is it because hoone who heard about it actually checked it out?
  • Is it because I've just done a really bad job of explaining what the system could do?
  • Is it because the current interface is so ugly that everyone's first reaction is to turn away in disgust?
  • Or is it because of something else I haven't thought of?

Please... I need feedback on this.

  • If this comment is so long that you're not going to bother reading the rest of it, please post a comment here saying so.
  • If you think that something like this may be a good idea, but don't feel like bothering to check out how the system works so far, please post a comment here saying so.
  • If you tried to check it out, but got lost because the interface is confusing, please post a comment here saying what you got confused by.
  • If you think this system is a bad idea, please post a comment here explaining why.
  • If you think the system is a good idea, but don't plan to try to help with the project, please post a comment here saying so.
  • If you think the system is a good idea, and you do plan to help with the project, please post a comment here saying so.

I had considered posting each of those suggestions as separate subcomments, to make the comments thread less tangled, but I'm not expecting many people to even read this comment, so that seemed a bit too ambitious, and seemed like downvote-bait.

If you upvote this comment, please post a comment here saying why you upvoted.

I suppose that even a downvote would be of at least some use as feedback, but if you downvote this comment, please at least post a comment here saying why you're downvoting it.

Though I suspect that I'll end up getting some downvotes just because of the tone of this comment, or some other social/signaling/memetic thing.

Ok, I'm done being angry and insecure now. On to more useful things.

More details about how the system currently works:

Documentation on how the system works is available at the main page for the Scenarios Project.

There are plenty of nodes already set up that you can browse through to get an idea of how the system works.

More details about how the system would work, if revised to be used for argument maps:

There are a few changes that would need to be made in order to use this system for argument maps, but I think the same core system would still basically do what we want.

All of these arguments would be stored on the wiki, so points from one argument can easily be used in another argument.

We would need to add more node types:

  • Claim
  • more?

And we would need to add more connection types:

  • supports
  • opposes
  • requires
  • more?

Many of the existing node types would be useful for argument maps:

  • groups
  • people
  • scenarios
  • actions
  • more?

And many existing connection types would be useful for argument maps:

  • is a necessary condition for
  • increases the probability of
  • decreases the probability of
  • is in favour of
  • is opposed to
  • more?

We would need a method for naming the nodes. One option is to just number them, but then that would make it hard to reuse nodes in later arguments. A better way to name the nodes would be to give a summary of the claim in a few words. Connections between nodes would be automatically named according to the nodes they connect, and the type of the connection.

We would also want a script to automatically generate the wikipages for all the nodes, given the annotated text of the argument, like in this experiment

We would also want to implement some of the other features listed in the Debate tools wiki page

Specifically: Probabilities! We need a way for the system to work with probabilities!

Footnotes:

I chose PHP because MediaWiki is written in PHP, and I am likely going to want to add these scripts into the MediaWiki code itself.

If this all ends up actually working, and being useful, then eventually it might be worth checking if it would be possible to do all this using Google Wave, or something else, so that the collaborative editing can be done in real time.

Question: Should I tidy up this comment some more, and post it as a top-level post?

Comment author: Morendil 09 February 2010 06:55:58PM 3 points [-]

Tidy up: in favor. :)

This is of interest to me. I haven't expressed interest because I didn't know about it.

Previously (pre-LW in fact) I've had an interest in using Semantic Wikis for the purpose of controversy mapping. Basing this type of work on a wiki makes sense to me, either in the context of controversy mapping a la MACOSPOL or in the context of trying to clarify my own thinking about cryonics.

Although (perhaps because) I used to love programming for fun, I have become a strong skeptic of just going off and implementing features on the off chance that the result will be somewhat useful. The code that's easiest to debug is the code you never write. My default move when thinking about doing something with technology is, "What can we possibly do that doesn't require implementing anything new?"

So, I would gladly volunteer to start "porting" my map of the cryonics debate to a Scenarios Map format, and report back with any issues I encounter. Offhand I feel that creating Wiki pages "by hand" might be the bottleneck, but I'm up to testing that.

Comment author: PeerInfinity 09 February 2010 07:56:26PM *  1 point [-]

Yay, finally some feedback!

Thanks :)

Before I started the Scenarios project, I checked out the semantic wikis that were available, and concluded that none of them were able to do what I was trying to do with the scenarios project.

Though I might as well admit that part of the reason why I went ahead and started coding this was just for fun.

Thanks for offering to help try this out!

My Skype ID is PeerInfinity, please feel free to contact me any time I'm online.

Though we shouldn't need to make all of the wiki pages by hand. We should have some sort of automated tool that generates most of the content on the wiki pages from an annotated chat log, like that experiment I tried.

I think the way this tool should work is that it should scan a wiki page containing the annotated conversation, and generate an XML file in the format used by MediaWiki's import tool. Then someone with admin access to the wiki can import the XML file using the import tool, or a regular user can manually copy the data from the XML file to the wiki.

Have you used PHP before?

...

And as for posting this idea as an actual post...

Maybe before I do that, I should make some of the changes I mentioned to the project, and actually try using it on an example argument or two.

I guess I'll use your cryonics argument and that other example argument.

...

Thinking about this project some more, it seems like these argument maps will fit in nicely with the scenarios map. Maps of arguments that a scenario will happen will fit in nicely with maps of things that could cause or prevent the scenario.

Or maybe if there are too many links, then everything will look all tangled... I guess one way to find out is to try it and see what happens...

Comment author: matt 16 February 2010 07:55:05PM 2 points [-]

It is my humble opinion that these tools need to start with user interface design. If it looks like a Steve Jobs Apple product and it has the right features, then it has a good chance of succeeding. I'd love this not to be the case.

Comment author: Roon 27 July 2010 02:32:42PM 0 points [-]

I'm surprised more tools to do this kind of thing don't already exist. It reminds me of the Truth Maintenance Systems I learned about in AI classes in the mid-90s.

Comment author: SilasBarta 27 July 2010 02:36:12PM *  2 points [-]

Why would academics want to make their field transparent to outsiders? ;-)

Comment author: PhilGoetz 07 February 2010 04:57:16PM 0 points [-]

Don't think about the output at all at this stage. The annotations should be XML. The output should be left to the discretion of the browser.