Follow-up to: Argument Maps Improve Critical Thinking, Software Tools for Community Truth-Seeking
We are here, among other things, in an attempt to collaboratively refine the art of human rationality.
Rationality is hard, because the wetware we run rationality on is scavenged parts originally intended for other purposes; and collaboration is hard, I believe because it involves huge numbers of tiny decisions about what information others need. Yet we get by, largely thanks to advances in technology.
One of the most important technologies for advancing both rationality and collaboration is the written word. It affords looking at large, complex issues with limited cognitive resources, by the wonderful trick of "external cached thoughts". Instead of trying to hold every piece of the argument at once, you can store parts of it in external form, refer back to them, and communicate them to other people.
For some reason, it seems very hard to improve on this six-thousand-year-old technology. Witness LessWrong itself, which in spite of using some of the latest and greatest communication technologies, still has people arguing by exchanging sentences back and forth.
Previous posts have suggested that recent software tools might hold promise for improving on "traditional" forms of argument. This kind of suggestion is often more valuable when applied to a real and relevant case study. I found the promise compelling enough to give a few tools a try, in the context of the recent (and recurrent) cryonics debate. I report back here with my findings.
I. Argunet
The first tool I tried was Argunet, an Open Source offering from the Institute of Philosophy in Berlin. I was seduced by the promise of reconstructing the logical structure of an argument, and by the possiblity of collaborating online with others on an argument.
Like other products in that category, the basic principle of operation of Argunet is that of a visual canvas, on which you can create and arrange boxes which represents statements, portions of an argument. Relationships between parts of an arguments are then materialized using links or arrows.
Argunet supports two types of basic relationship between statements, Supports and Attacks. It also supports several types of "inference patterns".
Unfortunately, when I tried using the Editor I soon found it difficult to the point of being unusable. The default expectation of being able to move boxes around by clicking and dragging is violated. Further, I was unable to find any way to move my boxes after initially creating them.
I ended up frustrated and gave up on Argunet.
II. bCisive Online
I had somewhat better luck with the next tool I tried, bCisive online. This is a public beta of a commercial offering by Austhink, the company already referenced in the previous posts on argument mapping. (It is a spin-off of their range of products marketed for decision support rather than argument support, but is also their only online, collaborative tool so far.)
The canvas metaphor proved to be implemented more effectively, and I was able in a relatively short time to sketch out a map of my thinking about cryonics (which I invite you to browse and comment on).
bCisive supports different types of statements, distinguished by the icons on their boxes: questions; arguments pro or con; evidence; options; "fixes", and so on. At present it doesn't appear to *do* anything valuable with these distinctions, but they proved to be an effective scheme for organizing my thoughts.
III. Preliminary conclusions
I was loath to invest much more time in updating my cryonics decision map, for two reasons. One is that what I would like to get from such a tool is to incorporate others' objections and counter-objections; in fact, it seems to me that the more valuable approach would be a fully collaborative effort. So, while it was worthwhile to structure my own thinking using the tool, and (killing two birds with one stone) that served as a test drive for the tool, it seems pointless to continue without outside input.
The other, more important reason is that bCisive seems to provide little more than a fancy mindmapping tool at the moment, and the glimpse I had of tool support for structuring a debate has already raised my expectations beyond that.
I have my doubts that the "visual" aspect is as important as the creators of such software tools would like everyone to think. It seems to me that what helped focus my thinking when using bCisive was the scheme of statement types: conclusion, arguments pro and con, evidence and "fixes". This might work just as well if the tool used a textual, tabular or other representation.
The argument about cryonics is important to me, and to others who are considering cryonics. It is a life decision of some consequence, not to be taken lightly and without due deliberation. For this reason, I found myself wishing that the tool could process quantitative, not just qualitative, aspects of my reasoning.
IV. A wish list for debate support
Based on my experiences, what I would look for is a tool that distinguishes between, and support the use of:
- a conclusion or a decision, which is to be "tested" by the use of the tool
- various hypotheses, which are offered in support or in opposition to the conclusion, with degrees of plausibility
- logical structure, such as "X follows from Y"
- challenges to logical structure, such as "X may not necessarily follow from Y, if you grant Z"
- elements of evidence, which make hypotheses more or less probable
- recursive relations between these elements
The tool should be able to "crunch numbers", so that it gives an overall indication of how much the total weight of evidence and argumentation contributes to the conclusion.
It should have a "public" part, representing what a group of people can agree on regarding the structure of the debate; and a "private" part, wherein you can adduce evidence you have collected yourself, or assign private degrees of belief in various statements.
In this way, the tool would allow "settling" debates even while allowing disagreement to persist, temporarily or durably: you could agree with the logical structure but allow that your personal convictions rationally lead you to different conclusions. Highlighting the points of agreement and contention in this way would be a valuable way to focus further debate, limiting the risk of "logical rudeness".
Comment in a nutshell: "Practice with very simple tools optimized for swift use in a culture that values logic and evidence are likely to be better than specialized tools with hard coded abstractions."
There's a specialized form of note taking called "flowing" within the policy/CEDA/NDT debate community. Here is a wikipedia article on the subject if anyone wants to hunt in the concept space for keywords and links and such :-)
In this speech community (especially at the higher levels) people tend to speak very swiftly because there is a background theory that "dropping arguments is conceding arguments" which creates an incentive to make many parallel arguments with the same conclusion that reach towards different sets of evidence. It is possible to win simply by making many adequate arguments that your opponent is incapable of handling with enough speed or concision.
(It can be dangerous to spin out many arguments without paying attention to how they interact, because sometimes a response will be something like "I concede argument 2 and 5 which constitute a turn on the general position that works like X. Arguments 1, 3, and 4 were about A, B, and C which are all less important than this, so I win the position.")
In terms of basic mechanics, the note taking process that I learned used strikingly simple tools: a four color pen, a legal pad, and (if you were going to prep for standard positions or responses by taking re-usable notes on what you would say) cover up tape. The process involved taking notes in a very narrow up-down column (about the width of cover up tape) so that iterated response and counter response could move horizontally across the page in rows. Time through each speech flows down on a given page, speech to speech flows horizontally. Arrows sometimes connect points if you underestimate the space requirements for a "clean flow" to help show how the rows are supposed to line up.
An eight minute speech would be flowed on several different pages, usually called "positions". As a matter of helping the judge flow the round (so they can take clean notes on your arguments and will notice when you win) it was wise to start a speech by announcing how many new positions would be introduced and/or the order in which existing positions would be addressed. Then everyone shuffles papers for the first few seconds of the speech. For a debate round on cryonics there might be a positions (each with a separate piece of paper) for "personal friendships and relationships", "health-dollar utility", "dehumanization critique", and so on.
Sometimes a column on a page would be a simple list of numbered points, but sometimes an argument would actually have an implicit tree structure, with mulitple points having sub points of their own. Comparing columns, then, its best to imagine one serialized tree next aligned with another. Structured argument was much more common in the early speeches of a round when significant preparation was possible and evidence was being read whose implications would be argued later on. Also, each piece of paper functioned sort of like its own "major tree branch" for the entire round (if you're trying to map the physical objects to an abstract data structure) and it was rare for cross applications of logic to jump from one page to another, but more common within a page.
The actual note taking would be full of your own jottings for arguments, like I used to use a capital T with a circle around it for "turn" which meant that the argument related to the "sign" (positive or negative) of the general position. An extended example might help...
Imagine that one side argues that the spread of some public perception would have a terrible consequence; then the other side might concede at the consequence level but proceed to argue that the perception already exists in the status quo and is growing and would actually go down on the basis of a proposed course of action... that would be a turn. In notes I might have one side flowed as "2. Perc {right arrow} H" (the 2nd argument was that perception is causally connected to harm) and then the response would be recorded just to the right with something like "T: Perc {up arrow}ing SQ" (perception is increasing in the status quo and somehow the proposed policy would decrease it, changing the sign of the broader argument).
Another important thing for the actual debate was that people would read sourced evidence into the round (at very high speed) so aside from the gist of the content, the author and year of publication could be very helpful because you might be familiar with them and have a retraction (or rebuttal by another author) written two years later in your own debate files. Even if you have no response you can go to the library with the citation and prep for the next time you see that evidence in a round... or you could use it yourself to argue the other side if it's just a great position :-)
One issue is that note taking systems like this work much better when the speakers are trying to be logically intelligible and expect you to flow them. The speakers can "signpost" as they raise a point by saying things like "...and my fourth argument here is that X" and then it can be responded with "their fourth argument about X is spurious because of Z". The numbers in the speech and in the notes help everyone retain the same alignment.
Sometimes I would debate using flowing skills in competitive contexts that were focused on rhetoric rather than reason (less signposting, less note-taking by the judge, speaker points awarded for wardrobe, little tolerance for meta-arguments about debate theory to enforce compliance with social norms, etc) and the note taking system generally still worked.
If the audience has a phobic reaction to stuttered logic and reason you can get hurt relying on the cultural forms of policy debate (in policy debate, T-shirts could reverse-signal to some judges that you relied on "pure logic"), but in the absence of phobias, "flow skills" still appeared to be relatively valuable because they helped you address people's points directly (which audiences usually like) and notice when they tried a logical shenanigan (which academic audiences enjoy seeing called out and given a latin name).
It's been many years since I judged or coached, and even more since I debated myself. My understanding is that in modern times people frequently bring laptops into rounds and flow using spreadsheets. Evidence is stored on hard drives instead of lugging around 100 pounds of paper in a filing system. It sounds altogether more civilized in that respect -- and learning to use the hotkeys for high speed note taking in spreadsheets is probably a skill that would help for years after in contexts like finance and consulting :-)
In any case, these sorts of experiences have adjusted my priors on the subject of rationality prostheses to favor of practice and skill with generic tools (paper and pen, or spreadsheets) as opposed to highly specialized software with a bunch of YAGNI features.
If any readers are in college (or better yet choosing a college), I would recommend googling around to see if you can find an associated intercollegiate policy debate team. Sometimes they are called CEDA (Cross Examination Debate Association) and sometimes NDT (National Debate Tournament) although the formerly distinct communities had already pretty well merged 10 years ago when I was debating. It can cost 20 to 60 hours a week, but I know of no other "game" with as wonderful a mix of adrenaline, logic, and language. In retrospect, I think that policy debating was more educational than many of my actual classes :-)
Update: in modern years, a tool called Verbatim exists, that allows for flowing to be done in Excel. The drawback is that you lose some flexibility, because you can't draw freeform arrows or unconventional symbols. The benefit is that you can type faster than you can write, you can copy and paste, you can keep files organized more easily than physical papers, and there are lots of community tools and support and macros. There are also some electronic tools for speech writing that let you easily import cards from various files into word, quickly reassemblin... (read more)