Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: Stuart_Armstrong 24 February 2017 10:06:34AM 3 points [-]

Thanks! Do you have a link to the original article?

Comment author: Gram_Stone 24 February 2017 12:40:48PM *  3 points [-]

The quote is from this article, section 4.1. There might be other descriptions elsewhere, Lenat himself cites some documents released by the organization hosting the wargame. You might want to check out the other articles in the 'Nature of Heuristics' series too. I think there are free pdfs for all of them on Google Scholar.

Comment author: Gram_Stone 23 February 2017 03:02:28PM 3 points [-]

Recently in the LW Facebook group, I shared a real-world example of an AI being patched and finding a nearby unblocked strategy several times. Maybe you can use it one day. This example is about Douglas Lenat's Eurisko and the strategies it generated in a naval wargame. In this case, the 'patch' was a rules change. For some context, R7 is the name of one of Eurisko's heuristics:

A second use of R7 in the naval design task, one which also inspired a rules change, was in regard to the fuel tenders for the fleet. The constraints specified a minimum fractional tonnage which had to be held back, away from battle, in ships serving as fuel tenders. R7 caused us to consider using warships for that purpose, and indeed that proved a useful decision: whenever some front-line ships were moderately (but not totally) damaged, they traded places with the tenders in the rear lines. This maneuver was explicitly permitted in the rules, but no one had ever employed it except in desperation near the end of a nearly-stalemated battle, when little besides tenders were left intact. Due to the unintuitive and undesirable power of this design, the tournament directors altered the rules so that in 1982 and succeeding years the act of 'trading places' is not so instantaneous. The rules modifications introduced more new synergies (loopholes) than they eliminated, and one of those involved having a ship which, when damaged, fired on (and sunk) itself so as not to reduce the overall fleet agility.

Comment author: skeptical_lurker 20 February 2017 10:08:35PM 8 points [-]

I think about politics far too much. Its depressing, both in terms of outcomes and in terms of how bad the average political argument is. It makes me paranoid and alienated if people I know join facebook groups that advocate political violence/murder/killing all the kulaks, although to be fair its possible that those people have only read one or two posts and missed the violent ones. But most of all its fundamentally pretty pointless because I have no desire to get involved in politics and I'm sure that wrt any advantages in terms of helping me to better understand human nature, I've already picked all the low hanging fruit.

So anyway, I'm starting by committing to ignore all politics for a week (unless something really earth-shattering happens). I'll post again in a week to say whether I stuck to it, and if I didn't, please downvote me to oblivion.

Oh, and replying to replies to this post are excepted from this rule.

Comment author: Gram_Stone 21 February 2017 10:57:57PM 0 points [-]

Why do you mourn when you can contemplate politics no more? What makes you think about it so much in the first place? That just seems like something you wouldn't want to ignore.

Comment author: lifelonglearner 21 February 2017 01:44:51AM 1 point [-]

Okay, gotcha. Thanks for the clarification on the points.

I admit I don't quite understand what MINERVA-DM is...I glanced at the paper briefly and it appears to be a...theoretical framework for making decisions which is shown to exhibit similar biases to human thought? (With cells and rows and ones?)

I'm definitely not strong in this domain; any chance you could summarize?

Comment author: Gram_Stone 21 February 2017 05:24:51PM 0 points [-]

I admit I don't quite understand what MINERVA-DM is...I glanced at the paper briefly and it appears to be a...theoretical framework for making decisions which is shown to exhibit similar biases to human thought? (With cells and rows and ones?)

I can't describe it too much better than that. The framework is meant to be descriptive as opposed to normative.

A complete description of MINERVA-DM would involve some simple math, but I can try to describe it in words. The rows of numbers you saw are vectors. We take a vector that represents an observation, called a probe, along with all vectors in episodic memory, which are called traces, and by evaluating the similarity of the probe to each trace and averaging these similarities, we obtain a number that represents a global familiarity signal. By assuming that people use this familiarity signal as the basis of their likelihood judgments, we can simulate some of the results found in the field of likelihood judgment.

I suspect that with a bit of work, one could even use MINERVA-DM to simulate retrospective and prospective judgments of task duration, and thus, planning fallacy.

Comment author: lifelonglearner 21 February 2017 12:55:54AM 1 point [-]

Hey Gram,

Thanks for the additional information!

I am assuming the first point is about this post and the second two are about the planning primer?

The feelings-as-information literature is new to me, and most of what I wrote here is from conversations w/ folks at CFAR. (Who, by the way, would probably be interested in seeing those links as well.)

I'll freely admit that the decision making part in groups was the weakest part of my planning primer. I'm not very sure on the data, so your additional info on improved group hypothesis generation is pretty cool.

There are definitely several papers on memory bias affecting decisions, although I'm unsure if we're talking about the same thing here. What I want to say is something like "improperly recalling how long things took in the past is a problem that can bias predictions we make" and this phenomena has been studied several times.

But there is also a separate thing where "in observed studies of people planning, very few of them seem to even use their memories, in the sense of recalling past information, to create a reference class and use it to help them with their estimates for their plans", which might also be what you're referring to.

Comment author: Gram_Stone 21 February 2017 01:17:49AM *  1 point [-]

I am assuming the first point is about this post and the second two are about the planning primer?

The first two are about this article and the third is about the planning fallacy primer. I mentioned hypothesis generation because you talked about 'pair debugging' and asking people to state the obvious solutions to a problem as ways to increase the number of hypotheses that are generated, and it pattern matched to what I'd read about hypothesis generation.

There are definitely several papers on memory bias affecting decisions, although I'm unsure if we're talking about the same thing here. What I want to say is something like "improperly recalling how long things took in the past is a problem that can bias predictions we make" and this phenomena has been studied several times.

I'm definitely talking about this as opposed to the other thing. MINERVA-DM is a good example of this class of hypothesis in the realm of likelihood judgment. Hilbert (2012) is an information-theoretic approach to memory bias in likelihood judgment.

I'm just saying that it looks like there's a lot of fruit to be picked in memory theory and not many people are talking about it.

Comment author: Gram_Stone 21 February 2017 12:23:07AM 1 point [-]

There is a problem where I say "Your hypothesis is backed by the evidence," when your entirely verbal theory is probably amenable to many interpretations and it's not clear how many virtue points you should get. But, I wanted to share some things from the literature that support your points about using feelings as information and avoiding miserliness.

First, there is something that's actually just called 'feelings-as-information theory', and has to do with how we, surprise, use feelings as sources of information. 'Feelings' is meant to be a more general term than 'emotions.' Some examples of feelings that happen to be classified as non-emotion feelings in this model are cognitive feelings, like surprise, or ease-of-processing/fluency experiences; moods, which are longer-term than emotions and usually involve no causal attribution; and bodily sensations, like contraction of the zygomaticus major muscles. In particular, processing fluency is used intuitively and ubiquitously as a source of information, and that's the hot topic in that small part of cognitive science right now. I have an entire book on that one feeling. I did write about this a little bit on LW, like in Availability Heuristic Considered Ambiguous, which argues that Kahneman and Tversky's availability heuristic can be fruitfully interpreted as a statement about the use of retrieval fluency as a source of information; and Attempts to Debias Hindsight Backfire!, which is about experiments that manipulate fluency experiences to affect people's retroactive likelihood judgments. The idea of 'feelings as information' looks central to the Art.

There is also a small literature on hypothesis generation. See the section 'Hypothesis Generation and Hypothesis Evaluation' of this paper for a good review of everything we know about hypothesis generation. Hardly inspiring, I know. The evidence indicates that humans generate relatively few hypotheses, or we may also write, humans have impoverished hypothesis sets. Also in this paper, I saw studies that compare hypothesis generation between individuals and groups of various sizes. You're right that groups typically generate more hypotheses than individuals. They also tried comparing 'natural' and 'synthetic' groups, natural groups are what you think; the hypothesis sets of synthetic groups are formed from the union of many individual, non-group hypothesis sets. It turns out that synthetic groups do a little better. Social interaction somehow reduces the number of alternatives that a group considers relative to what the sum of their considerations would be if they were not a group.

Also, about your planning fallacy primer, I think the memory bias account has a lot more going for it than a random individual might infer from the brevity of its discussion.

Comment author: RomeoStevens 28 January 2017 09:46:05PM 4 points [-]

The executive summary would be that TAPs are cfar discovering one of these. You can hit various systems with the decomposition hammer and you'll start to see the more common pieces crop up over and over. OODA loop, GTD, analogical reasoning, sorting schemes for prioritization. The tell tale sign of one of these is that you can feed it to itself, which indicates it is flexible enough to take all sorts of arguments.

I'll try to write a short post on it at some point.

Comment author: Gram_Stone 28 January 2017 10:53:54PM 0 points [-]

I'll try to write a short post on it at some point.

Please do!

Comment author: Gram_Stone 26 January 2017 10:23:14PM 5 points [-]

(Tentatively upvoted.)

I find that a good way to make statements criticizing individuals or organizations less provocative is to frame your criticism as a confusion. This simultaneously allows you to demonstrate that you've thought about their reasoning for more than five minutes and tends to make any further discussion less adversial.

The abstract reasoning about why prison reform is a bipartisan cause makes sense to me: prisons cost lots of money (bad conservative metric) and they're disproportionately inhabited by minorities (bad liberal metric), but if your descriptions of their recommended organizations are charitable, then I too am confused right now.

Comment author: Gram_Stone 24 January 2017 02:46:05AM 0 points [-]

Does anyone have an electronic copy of the Oxford Handbook of Metamemory that they're willing to share?

Comment author: Gram_Stone 22 January 2017 04:17:32AM *  4 points [-]

I think it's possible to exercise Hufflepuff virtue in the act of encouraging more Ravenclaw virtue, right? That is, getting an arbitrary ball rolling is a Hufflepuff thing to do, even if you roll the ball in a Ravenclaw direction? That's an important distinction to me.

A mid-term goal of mine is to replicate Dougherty et al.'s MINERVA-DM in MIT/GNU Scheme (it was originally written in Pascal; no, I haven't requested the authors' source code, and I don't intend to). I also intend to test at least one of its untested predictions using Amazon Mechanical Turk, barring any future knowledge that makes me think that I won't be able to obtain reliable results (which has only become less plausible as I've learned more; e.g. Turkers are more representative of the U.S. population than the undergraduate population that researchers routinely sample from in behavioral experiments; there's also a few enthusiasts who have done some work on AMT-specific methodological considerations).

MINERVA-DM is a formal model of human likelihood judgments that successfully predicts the experimental findings on conservatism, the availability heuristic, the representativeness heuristic, the base rate fallacy, the conjunction fallacy, the illusory truth effect, the simulation heuristic, and the hindsight bias. MINERVA-DM can also be described as a modified version of Bayes' Theorem. I'm not too far yet, having just started learning Scheme/programming-in-general, but I have managed to hobble together a one-line program that outputs an n-vector with elements drawn randomly with replacement from the set {-1, 0, 1}, so I guess I've technically started writing the program.

It's worth saying that I'm not very confident that MINERVA-DM won't be overturned by a better model, and that's not the point.

I need some sort of example, and MINERVA-DM has good properties as an example, because its math is exceedingly simple (i.e., capital-sigma notation, arithmetic mean, basic probability theory (see Bolstad's Introduction to Bayesian Statistics, Ch.3), etc. There are probably plenty of improvements that we need to and could make as a community, but my own concern is that it's never been winter-night-clear to me why at least some of us aren't trying to perform (Keyword Alert!) heuristics and biases/judgment and decision making (JDM)/behavioral decision theory research on LW or on whatever conversational focus we may be using in the near- to mid-term future. There is no organization in the community for this; CFAR is the closest thing to this, and AFAICT, they are not doing basic research into H&B/JDM/BDT. People around here seem to me more likely than most to agree that you're more likely to make progress on applications if you have a deep understanding of the problem that you're trying to solve.

I think it is intuitive that you simply cannot productively do academic work solely in the blogosphere, and when you're explaining a counterintuitive point, a point that is not universal background knowledge, you should recurse far enough to prevent misunderstandings in advance. I no longer find it intuitive that you can't do a substantial amount of work on the blogosphere. For one, a good deal of academic work, especially the kind we're collectively interested in, doesn't require any special resources. Reviews, syntheses, analyses, critiques, and computational studies can all be done from a keyboard. As for experiments, we don't need to buy a particle accelerator for psych research, you guys; this is where Mechanical Turk comes in. E.g. see these two blog posts wherein a scientist replicates one of Tversky and Kahneman's base rate fallacy results with N = 66 for US$3.30, and replicates one of Tversky and Kahneman's conjunction fallacy results with N = 50 for US$2.50. (Here's a list with more examples.)

Arguing that there's important academic work that doesn't require anything but a computer (reviews, syntheses, analyses, computational studies), and demonstrating that you can test experimental predictions with your lunch money seems like a good start on preempting the 'you can't do real science outside of academia' criticism. (It's not like there isn't a precedent for this sort of thing around here anyway.) It also prevents people from calling you a hypocrite for proposing that the community steer in a certain direction without your doing any of the pedaling. I probably would've kept quiet for a lot longer if I didn't think it were important to the community to respond to calls like this article, especially considering that we may be moving to a new platform soon.

View more: Next