The post cites being upset or angry as evidence of certain apparent preferences being closer to genuine preferences, but a paperclip maximizer wouldn't get upset or angry if a supernova destroyed some of its factories, for example. I think being upset or angry when one's consciously held goals have been frustrated is probably just a signaling mechanism, and not evidence of anything beyond the fact that those goals are consciously held (or "approved" or "endorsed").