In art circles, there’s a common piece of advice that goes along the lines of:
When people say that they don’t like something about your work, you should treat that as valid information.
When people say why they don’t like it or what you could do to fix it, you should treat that with extreme skepticism.
This is also very, very true in UX design (and all similar fields such as print design, etc.).
Edit: This is why “I didn’t like X” or “X seems ugly” or “I have a hard time reading X” is extremely valuable feedback, and any designer is always happy to hear it. On the other hand, “X is designed wrong because [criticism of specific design decision]” is basically worthless feedback, and almost never helps in any way.
Edit 2: Note that the above is the opposite of what people’s intuitions tell them constitutes valuable feedback. Non-designers often think that “I didn’t like it” or “it’s ugly” is unhelpful, and they try to be more helpful by making specific criticisms (like “the text is justified; it shouldn’t be”). Coming from a layperson, this attempt to be helpful is actually the diametric opposite of an improvement, turning useful feedback into useless advice.
Edit 3: The most useful feedback is the one that tells me what is the specific problem you are experiencing. The subjective nature of the feedback is important!
It's especially bad in video games. Any popular game will have many reviews with clueless game design advice, instead of immediate reactions. That's one reason watching playtesters can be more useful than asking for feedback.
Indeed, user testing is far superior to asking for “feedback” (and this is also the reason why conducting user tests is a non-trivial skill set—the tests must be set up properly, and users must be prepared, prompted, and guided properly, so as to elicit reactions and not evaluations or suggestions etc.).
Of course, one does regularly receive user feedback outside of the context of formal user testing, and such feedback can be quite useful (especially if it’s spontaneous)—but not, unfortunately, if the “pure subjective reactions aren’t helpful; try to give suggestions” meme is in effect.
I haven't actually put too much into this sub-question, curious about your thoughts: is it actually that important that users don't attempt to give feedback, or merely that the feedback-receiver makes sure to translate in their head all instances of "this is bad because Y, you should do X because Z" into "I didn't like this experience, something about it is off?"
The problem with such translation attempts is that information is lost.
The thing is that comments of the form “X is bad because Y” can be prompted by any number of things.
Some of those are some form of bad experience that user had with your product. (However, you don’t know what sort of bad experience; the given reasons why X is supposedly bad are, as noted in the OP, generally confabulated.)
But some of those possible triggers for “X is bad because Y” comments are not any specific bad experience the user had, but rather some belief the user picked up about X being bad, some guideline gleaned from some listicle somewhere, some vague association based on a passing fad, some outright misunderstanding, some attempt to give general advice, etc., etc.
So what I want to know is, in what way did the user attempt to interact with the product, and what difficulty, dissatisfaction, or other undesirable experience ensued? That is the critical data.
But if the answer turns out to be “none, really, I didn’t even try using your thing and have no desire to use it, I’m just saying that aspect X of your thing is bad”, then there’s not much reason to take that as a “user is dissatisfied” data point.
I wonder if the desire to justify is the result of social stigma surrounding being the wrong sort of person. If to be a full person is to have reasons for your actions, and to be nice to others, then it makes sense to me that the desire to give reasons for "mean" criticism is a result of wearing the correct model of personhood.
That, and the fact that when making decisions, it's *really important* to have non-subjective reasons -- or if you have subjective reasons, you still have objective reasons why they matter, like "if I don't like someone on a personal level, I really shouldn't spend the rest of my life with them" in dating.
So people are used to a mode of thought where a subjective opinion means "you're not done explaining"/"you haven't spent enough mental effort on the problem," and they engage the -- honestly, very productive, very healthy -- same mechanisms they use when justifying a command decision. It just happens to be mis-applied in this case.
An example from Feynman's «The Character of Physical Law»:
The next guy who did something great was Maxwell, who obtained the laws of electricity and magnetism. What he did was this. He put together all the laws of electricity, due to Faraday and other people who came before him, and he looked at them and realized that they were mathematically inconsistent. In order to straighten it out he had to add one term to an equation. He did this by inventing for himself a model of idler wheels and gears and so on in space. He found what the new law was – but nobody paid much attention because they did not believe in the idler wheels. We do not believe in the idler wheels today, but the equations that he obtained were correct. So the logic may be wrong but the answer right.
Especially for the study of consciousness and mental states, I associate this useful tactic with Daniel Dennett's term "heterophenomenology".
I also thought of heterophenomenology when reading this. (It is, I think, quite a useful concept, and it would be good if someone wrote an LW post explaining it. I will do it at some point, if no one else gets there first.)
Curated. For me, this post has stood the test of time with me thinking back to it with non-trivial regularity. This post moved me (often) from reacting to things with "that's definitely wrong!" to "but why are you saying it in the first place?" with the result of me noticing things I wouldn't have otherwise and also causing me to listen more carefully and patiently in ways that were good instead of tuning people out of the first epistemically offensive (imo) thing they said.)
Meta: it seems like you have accidentally added this comment also at the beginning of the post besides commenting?
Ah, I just realized this is the norm with curated posts. FWIW I feel a bit awkward to have the curation comments displayed so prominently, since it alters the author's intended reading experience in a way I find a bit weird / offputting.
If it was up to me, I would remove the curator's words at the top of a post in favor of comments like this one, where the reasons for curation are explained but its not the first thing that readers see when reading the post.
Yes I also find the curation tagline to be awkward. The first few times I saw it I assumed that the original author had edited their post to include the curation comment at the start. I only discovered this was not the case by reading the comments here. Seems that editing another user's post is bad form. Adding your own comment to the start of their post is especially obtrusive, but in this case obtrusive is way better than non-obtrusive so that's a plus . The current curated tag should be enough for the website itself. Maybe the subscription e-mail can retain the curation notice while the original post remains unedited.
Unless the original authors are contacted and agreed to the curation notice being included at the top of their posts. Then everything is fine as it is.
I contacted the authors the first couple of times while trying this new thing and neither objected, nor had anyone else yet, so I didn't check with Kaj (sorry Kaj).
It's good now that people have spoken about and we'll think about how to get the good without the bad. As Habryka mentioned, the thing we're trying to achieve is giving people context on the curation. I've wanted to avoid putting something prominently in front of people's faces without them knowing why. Not all curated posts require this, but sometimes we curated weird ones, or ones where it's being curated for the comments rather than the post itself, or ones that are irregular (e.g. important Covid updates). In all those cases, it feels important to provide an explanation that gets seen.
As Habryka mentioned, if we keep doing this, we'll build a way to do that doesn't involve editing. This was just the quick test version.
Thanks again though for the feedback!
You're doing good work with the curation and it's very effective at bringing important posts back into the reader's eye so thanks for that! I would probably have never seen this post otherwise. I'm glad you're working on the system to iron out the kinks.
One of the reasons for this is that curated posts get emailed out to a few thousand people and that email doesn't include any comments, and frequently others have asked us for context on why we curated something when they get the email. So this way the curation notice gets to be included in the email.
A minor change might make this more perspicuous (although this suggestion should be viewed in the light of the OP and the thread from Said Achmiz's comment on UX design): Begin the curation notice at the top of the OP with "Curated by (username)".
How hard is it to make the Curation admin tooling include a text field for curation reason, that both gets posted as a comment, and included in the email? This really seems like a nice-to-have software feature request, rather than something that merits the grotesque violence of editing the OP!
As mentioned elsethread, I checked with authors the first couple of times. Having met no objections, I applied induction and tested whether maybe the overhead wasn't necessary.
Shouldn't be too hard, but seemed worth testing whether it was working in a low-tech solution before we build a thing. My guess is we will add this more properly relatively soon, if we keep doing it.
+1
I have been unsure how long it could persist for. There's an argument for it being present all the time it's at the top of the curated list, and if you imagine author's being proud of it, for even longer than that. But perhaps (while we're doing the low-tech thing), it should only be for sending out the email.
I have done so but it was not an accident! Top of the post version makes it so it's easy to see the notice when you view the post and on email, the comment lets it show up in recent discussion, also discussion of the notice if people are inclined. So a bit of redundancy on purpose.
(Separately from the violence issue, it looks like "the community" has an initials collision: I initially read "RB" as "Rob Bensinger" rather than "Ruby Bloom.")
I've found myself referencing this post repeatedly since reading it. It's improved my reaction to ideas and models that seem definitely wrong. Now instead of just thinking "that's clearly wrong", I'm moved to ask "but what true observations are leading to this model?" and it feels like I see more value in even wrong models. Sometimes, I think, I learn to see where they're not wrong.
I also want to give this post credit for feeling quite original to me. Many other posts are refinements or clarifications of ideas which exists in some form elsewhere. While I can't be sure, this posts feels like it really cemented something new in me rather than just helping a thing I already knew about stick.
All in all, I really like this post.
Succubi/incubi and the alien abduction phenomenon point to hypnagogia, and evo-psych explanations of anthropomorphic cognition are often washed down with arguments that anthropomorphism causes good enough decisions while being technically completely false; there's an old comment by JenniferRM talking about how surprisingly useful albeit wrong it would be to model pathogens as evil spirits.
gosh, just the title succinctly expresses what I've spent multiple paragraphs trying to explain many times over. unusually good compression, thank you.
People have poor introspective access to the reasons why they like or dislike something; when they are asked for an explanation, they often literally fabricate their reasons.
omg, they literally work that way. I can't, let me off
Shout out to Karl Marx for correctly identifying many issues with what he called capitalism, but providing a compeltely inaccurate hypothesis as to the root cause of those issues.
IMO, Karl Marx's main errors come down to 2 errors:
Overestimating drastically the amount of zero-sum/win-lose interactions in the economy. To be a bit more fair, the notion that positive sum/win-win economics could exist as a supermajority of all trades was likely very undeveloped by Marx's time, and it only got started by at most 20-50 years in the west. This is a common error made across the political spectrum, but Karl Marx's assumption that there had to be conflict between the proletariat and the bourgeoisie probably reflects zero-sum assumptions on economics.
Way overestimating how much change/radicalism was necessary to distribute some of the profits of the new capitalism economy, closely related to the zero sum error, but also not realizing that the rich weren't omnipotent and couldn't just arbitrarily change society to fit their preferences.
1. The Consciousness Researcher and Out-Of-Body Experiences
In his book Consciousness and the Brain, cognitive neuroscientist Stansilas Dehaene writes about scientifically investigating people’s reports of their out-of-body experiences:
The naive hypotheses that out-of-body experiences represented the spirit genuinely leaving the body, were incorrect. But they were still pointing to a real observation, namely that there are conditions which create a subjective experience of leaving the body. That observation could then be investigated through scientific means.
2. The Artist and the Criticism
In art circles, there’s a common piece of advice that goes along the lines of:
When people say that they don’t like something about your work, you should treat that as valid information.
When people say why they don’t like it or what you could do to fix it, you should treat that with some skepticism.
Outside the art context, if someone tells you that they're pissed off with you as a person (or that you make them feel good), then that's likely to be true; but the reason that they give you may not be the true reason.
People have poor introspective access to the reasons why they like or dislike something; when they are asked for an explanation, they often literally fabricate their reasons. Their explanation is likely false, even though it’s still pointing to something in the work having made them dislike it.
3. The Traditionalist and the Anthropologist
The Scholar’s Stage blog post “Tradition is Smarter Than You Are“, quotes Joseph Henrich’s The Secret of Our Success which reports that many folk traditions, such as not eating particular fish during pregnancy, are adaptive: not eating that fish during pregnancy is good for the child, mother, or both. But the people in question often do not know why they follow that tradition:
The people’s hypotheses for why they do something is wrong. But their behavior is still pointing to the fish in question being bad to eat during pregnancy.
4. The Martial Artist and the Ki
In Types of Knowing, Valentine writes:
The explanation about magical life energy was false, but it was still pointing to a useful trick that could be learned and put to good use.
Observations and the hypotheses developed to explain them often get wrapped up, causing us to evaluate both as a whole. In some cases, we only hear the hypothesis rather than the observation which prompted it. But people usually don’t pull their hypotheses out of entirely thin air; even an incorrect hypothesis is usually entangled with some correct observations. If we can isolate the observation that prompted the hypothesis, then we can treat the hypothesis as a burdensome detail to be evaluated on its own merits, separate from the original observation. At the very least, the existence of an incorrect but common hypothesis suggests to us that there’s something going on that needs to be explained.