= 27d567bc2d48d9f40e9ccc20ea370b15)
pjeby comments on SotW: Check Consequentialism - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (311)
Cleverness-related failure mode (that actually came up in the trial unit):
One shouldn't try too hard to rescue non-consequentialist reasons. This probably has to be emphasized especially with new audiences who associate "rationality" to Spock and university professors, or audiences who've studied pre-behavioral economics, and who think they score extra points if they come up with amazingly clever ways to rescue bad ideas.
Any decision-making algorithm, no matter how stupid, can be made to look like expected utility maximization through the transform "Assign infinite negative utility to departing from decision algorithm X". This in essence is what somebody is doing when they say, "Aha! But if I stop my PhD program now, I'll have the negative consequence of having abandoned a sunk cost!" (Sometimes I feel like hitting people with a wooden stick when they do this, but that act just expresses an emotion rather than having any discernible positive consequences.) This is Cleverly Failing to Get the Point if "not wanting to abandon a sunk cost", i.e., the counterintuitive feel of departing from the brain's previous decision algorithm, is treated as an overriding consideration, i.e., an infinite negative utility.
It's a legitimate future consequence only if the person says, "The sense of having abandoned a sunk cost will make me feel sick to my stomach for around three days, after which I would start to adjust and adapt a la the hedonic treadmill". In this case they have weighed the intensity and the duration of the future hedonic consequence, rather than treating it as an instantaneous infinite negative penalty, and are now ready to trade that off against other and probably larger considerations like the total amount of work required to get a PhD.
My normal response is, "so what's bad about that?" and go a few rounds until the person has to struggle for an answer... the teachable moment where I can say, "you see what you're doing? you're just making stuff up. What's actually going to happen?"
(That being said, it would definitely have been helpful for me in the past if I had thought to confine questions of consequences to things happening at a point-in-time. I eventually figured out that I needed to ask that for things people were thinking about or remembering, but there was a long time where I also had the hit-them-with-a-stick frustration to this kind of response.)
The only suggestion I have for exercises is to make people write down their own thinking (or state their thinking out loud), and then read it back as a kind of grammar-checking exercise. Are these abstract nouns or concrete nouns? Do they describe a point in time or some sort of vague non-timey thing?
I've done some similar things with small groups, though, and one thing that becomes quickly apparent is that everybody already knows when somebody else is doing it wrong. The part of the exercise that's hard, is learning to apply it to your own thoughts or utterances, and for that, it helps to externalize them first, then treat them as input.
To put it another way, the prerequisite 5-second skill for consequence checking is reflecting on what you just said or thought. If people don't reflect on their utterances, no further debiasing skills can be applied.