I wonder if this attributing revealed preference to a person rather than a person-situation pair should set off fundamental attribution error alarms.

Good plausible hypothesis to cache for future priming, but I'm not sure I fully understand it:

preferences someone reveals through programming them into a transhuman AI

More specifically, what process are you envisioning here (or think others might be envisioning)?