With respect to situations where people are complaining or being critical of a policy or a behavior, one should bear in mind that a compliant is more likely to be voiced than a neutral or positive attitude about the same thing. As a TV writer, I can tell you that the amount of critical 'hate-mail' I receive about an episode often conflicts with the audience polling results for the episode. My 'hate-mail' might outpace other messages by two to one for an episode which polls in the overwhelmingly positive range. I suspect that this comes down to how much more likely a person is to act on a negative opinion than on a positive one.
A point I think others missed here is that in the TV example, there's more data than the situations the OP talks about, so mscottveach can say there's a disparity instead of just having the hatemail. Maybe more situations should involve anonymous polling.
Interesting point! I think what tends to be voiced vs. not voiced varies a lot based on both the field and the culture involved. I've been in some environments where it seems like everyone loves to complain even when things are fine, but I've also been in some where people are very reticent to speak up even when there's a problem.
I wonder if there are useful statistics anywhere on how this applies to different areas, as this seems like something that might be helpful to take into account when thinking about how best to process feedback.
I think what tends to be voiced vs. not voiced varies a lot based on both the field and the culture involved. I've been in some environments where it seems like everyone loves to complain even when things are fine, but I've also been in some where people are very reticent to speak up even when there's a problem.
I'm not sure that's at odds with what mscottveach is saying. To put it in different words, while the amount of feedback might vary, I don't think the ratio of positive vs. negative feedback varies. It's the very rare situation where the number of messages that say, "This was good, everything went as planned or intended," outnumbers the messages that talk about how something went wrong.
I'm not sure that's at odds with what mscottveach is saying. To put it in different words, while the amount of feedback might vary, I don't think the ratio of positive vs. negative feedback varies. It's the very rare situation where the number of messages that say, "This was good, everything went as planned or intended," outnumbers the messages that talk about how something went wrong.
Oh, I quite disagree. I've often found it normal for people to give positive feedback only or for positive feedback to far outweigh negative. In fact, I'm a little surprised to hear you say this because my experience has often been the opposite -- that it can be rare and difficult to get people to give negative feedback or genuine criticism of something!
For instance, when I've helped organize parties or social events I've IIRC heard almost only positive things, which is not to say that my parties and events were astoundingly good but rather that the norms favor positive over negative feedback there.
Similarly, I recently started up an online ladder for players of a game that I like, and the feedback there has been quite positive as well. I don't think I did a superb job in doing that -- I actually released it months after I'd been planning to -- but people tend to give positive feedback on those sorts of projects.
In fact, I've thought before that this positive bias in feedback can often be an obstacle to progression, as flawed projects don't get corrected as easily if people don't point out the flaws -- note also Said's comment about how it can be difficult to get people to point out even egregious errors in software/web design!
Oh, yes, that's a good point, and I have updated in your direction. I was thinking more along the lines of things like product reviews and survey feedback, where the user is much more likely to take the time to complete the feedback form if they've had a negative experience than if they've had a positive one.
Edited to add: I wonder if there's a distinction between unsolicited feedback and requesting feedback, or giving feedback to an individual vs. feedback to a corporate entity.
where the user is much more likely to take the time to complete the feedback form if they've had a negative experience than if they've had a positive one.
Sounds like they're being used as bug reports. Is there usually another way of making bug reports?
This phenomenon is very familiar to me as a UX designer, because it often makes bugs or design flaws way, way more difficult to learn about than should (one would naively imagine) be the case.
Specifically: suppose I release some piece of software, which has some bug, or usability flaw, etc.; and suppose this problem does not manifest for me, in my own testing, but does manifest for many other users, on a regular basis. One might expect that a flood of complaints, bug reports, angry tweets, irate emails, etc., would nigh-instantly alert me to the problem’s existence… but instead there is naught but silence, and I remain blissfully unaware that there’s anything wrong.
Then some time passes, and—by sheer accident!—I discover that large numbers of users have been living with this problem for weeks or months or (horror!) years, and just haven’t said anything… because they’re used to technology just… not working very well, or having bugs, etc.; and so they shrug and treat it as “one of those things”, and do some workaround, or just tolerate the problem, and never consider that, actually, there is something wrong with this picture, and that it is possible for this problem (indeed, most such problems) to not exist, and that complaining might yield results.
As they say—many such cases! (Here, for example, is gwern tweeting about a case when a website he built had 460,000 unique visitors before word got to him before he realized, after checking personally, that the layout was broken on mobile devices!)
EDIT: Corrected the gwern anecdote—it was even worse than I remembered.
before word got to him that the layout was broken on mobile devices
Emphasizing the point even more - word didn't get to me. I just thought to myself, 'the layout might not be good on mobile. I ought to check.' (It was not good.)
… the situation was so disheartening that in my memory of it I mentally substituted something more palatable!
(Fixed, thanks.)
Closely related to this is the issue where people try to do something, fail, and figure that they will "try harder" the next time. Frequently this just means that they will fail again, because their understanding of what they are doing wrong isn't sufficiently gears-level to allow them to isolate the bug in question; "I will try harder" tends to mean "I don't know why exactly I failed, so I'll just try again and hope that it works this time around".
I did some peer coaching at one point, and a common thing was that one of us would make a plan in order to do Y; a week later, the plan had failed and Y remained undone. The one doing the coaching would then ask what went wrong and how the other person could fix that failure, producing a revised plan for next week. Often, drilling down would produce something specific, such as "I had planned to get exercise by going out on a run, but then I was busy on a few days and it rained on the rest". Then you could ask yourself what kind of a plan would avoid those failure modes, and generate a less fragile approach.
That makes "why do I keep having this problem" and "what have I tried before and why hasn't it worked" very useful questions, and might help reframe failure not as failure, but as progress towards solving the goal. Yes, you didn't succeed at the goal right away, but you got more information about what works and what doesn't, making you better-positioned to solve it the next time around.
This is also good to combine with Murphyjitsu - after forming a new plan, imagining that plan to have failed and asking yourself how surprising that possibility would feel. If it wouldn't feel very surprising at all, ask your brain what it expects the failure to have been caused by, and plan around that.
It's also worth noting that sometimes you go through a few iterations of this and new bugs just keep popping up, or alternatively it feels like you can't imagine anything in particular that would go wrong, but your inner simulator still expects this to fail. That might point to there being some emotional issue, such as your brain predicting that success will be dangerous for some reason. That's then another thing that you can try to tackle.
A relevant old comment of mine. (Dagon’s point is similar to half of what I say there.)
EDIT: This other old comment is also relevant.
Useful observation/reminder. I might generalize it to "there is a conflict between expectations/preferences of the users and the implementer". It's not necessarily the policy/door/game that is in the wrong; there may be legitimate but un-obvious or conflicted reasons for the choice. Sometimes users really do need to go through the education (and perhaps grieving) process in order to match reality better.
I very much sympathize with the person who's sick of having the same conversation, and it happens with good policies as well as with bad - there's no information about the quality of the policy in the complaints and confusion. There _IS_ evidence that the policy rollout process is flawed. Controversial or unpleasant policies need to brought out by senior management first, so they can answer questions (or punish troublemakers, depending on maziness of the org).
I didn't learn this until I was 24. Up until that point I always thought "abc is only happening because bad / stupid actors are doing ijk instead of xyz".
Trump getting elected was actually the precise moment that I realized placing expectations on reality is futile, and you have to attempt to perceive reality in the most unadulterated way possible.
There can be systematic reasons why a problem keeps happening which in itself is not a problem. For example sports judged might get yelled at a lot but that is mainly because sports fans are invested and being in a position of non-power can use the issue to emote safely. Does that mean the judge is doing something wrong? Probably not. Does it mean there is organizational dysfunction in the roles of judge and sports fan? Maybe but not neccesarily.
If something happens with statistical predictability it probably isn't due to random chance and that can be used as a que that it needs or can accept an explanation. And when one does pose an explanation one can be wrong about it.
Even if it is not an opportunity to improve the system/actor it can be a opportunity to learn more about it.
My first impression was that this is much too obvious to be worth talking about, but my second thought is that I've found it very useful to have these language based triggers that act as a "summon sapience" spell. Triggers that make you stop and think. They don't push you to make any particular choice, but just to notice that this is a situation where there is a choice to be made.
I wonder if it would be worth putting together a list of "words and phrases which, when you hear them, should make you stop and think". "Why does this keep happening?" belongs on that list for sure.
One thing I've noticed recently is that when someone complains about how a certain issue "just keeps happening" or they "keep having to deal with it", it often seems to indicate an unsolved problem that people may not be aware of. Some examples:
I'm not going to say that all such cases are ones where things should change -- it's certainly possible that one might have to take unpopular but necessary measures under some circumstances -- but I do think that this sort of thing should be a pretty clear warning sign that things might be going wrong.
Thus, I suspect you should consider these sorts of patterns not just as "some funny thing that keeps happening" or whatever, but rather as potential indicators of "bugs" to be corrected!
[1] This post was primarily inspired by a situation in which I saw someone write "This is the fifth time I've had this conversation in the last 24 hours and I'm sick of it" or words to that effect -- the reason they had kept having that conversation, at least in my view, was because they were implementing a bad policy and people kept questioning them on it (with perhaps varying degrees of politeness).