Too much feedback can be a bad thing

Kaj_Sotala

Didn't have the time to read the article itself, but based on the abstract, this certainly sounds relevant for LW:

Recent advances in information technology make it possible for decision makers to track information in real-time and obtain frequent feedback on their decisions. From a normative sense, an increase in the frequency of feedback and the ability to make changes should lead to enhanced performance as decision makers are able to respond more quickly to changes in the environment and see the consequences of their actions. At the same time, there is reason to believe that more frequent feedback can sometimes lead to declines in performance. Across four inventory management experiments, we find that in environments characterized by random noise more frequent feedback on previous decisions leads to declines in performance. Receiving more frequent feedback leads to excessive focus on and more systematic processing of more recent data as well as a failure to adequately compare information across multiple time periods.

Hat tip to the BPS Resarch Digest.

ETA: Some other relevant studies from the same site, don't remember which ones have been covered here already:

Threat of terrorism boosts people's self-esteem

The "too much choice" problem isn't as straightforward as you'd think

Forget everything you thought you knew about Phineas Gage, Kitty Genovese, Little Albert, and other classic psychological tales

Didn't have the time to read the article itself, but based on the abstract, this certainly sounds relevant for LW:

Recent advances in information technology make it possible for decision makers to track information in real-time and obtain frequent feedback on their decisions. From a normative sense, an increase in the frequency of feedback and the ability to make changes should lead to enhanced performance as decision makers are able to respond more quickly to changes in the environment and see the consequences of their actions. At the same time, there is reason to believe that more frequent feedback can sometimes lead to declines in performance. Across four inventory management experiments, we find that in environments characterized by random noise more frequent feedback on previous decisions leads to declines in performance. Receiving more frequent feedback leads to excessive focus on and more systematic processing of more recent data as well as a failure to adequately compare information across multiple time periods.

Hat tip to the BPS Resarch Digest.

ETA: Some other relevant studies from the same site, don't remember which ones have been covered here already:

Threat of terrorism boosts people's self-esteem

The "too much choice" problem isn't as straightforward as you'd think

Forget everything you thought you knew about Phineas Gage, Kitty Genovese, Little Albert, and other classic psychological tales

I haven't read the article, but the more noise there is, the more samples you should get before giving the feedback significance.

A recent Science article showed this graphically. They were using a two-deck game, where you give subjects two decks of cards, each with 2 types of cards, winning and losing. You tell them that one deck has more winning cards than the other. Then they get a certain number of card draws, and can draw each card from either deck.

What's amazing is how lousy people do on this test. You would win by drawing 10 cards from each deck, choosing the better deck, and sticking with it. People don't. (Received wisdom is that they want to draw from each deck in proportion to its probability of a winning card, which is not a good strategy.)

In this paper, they posited that people estimate the probability of a deck being the better deck by exponentially discounting older evidence. (The most recently drawn card influences them most.) They plotted a graph of the probability over time that this would give you of one deck being the winning deck. Amazingly, this graph showed the probability hovering around .5 over 90 card draws, when actually the difference between the decks was dramatic (something like .6 win vs. .4 win).

11

Too much feedback can be a bad thing

11

11

11

Too much feedback can be a bad thing

11

11