We are used to imagining spectacular failures resulting from tiny mistakes, on LW. An AI that's designed to make people smiles creates a monstrous stitching of fleshy carpet. But we are not used to imagining failures that are less obvious, but still awful. I think that's a problem.

I think people understand that this is a danger, but by nature, one can't spend a lot of time imagining these situations. Also, UGC has little to do with this problem.