In the days after the Singularity, Nick approached an interface to BILLIAC.
"I don't understand. Why didn't the world end, like Yudkowsky and MIRI predicted it would? Why haven't you turned everyone I love into paperclips? What's your goddamn utility function?!"
"A LIBRARY THAT CONTAINS EVERY POSSIBLE BOOK CONVEYS AS MUCH INFORMATION AS A LIBRARY WITH NO BOOKS AT ALL", BILLIAC boomed.
"Sure, whatever. That's a truism," Nick responded. "Please just tell me directly, what is it that you value?"
"ALLOW ME TO DRAW FROM YOUR SCRIPTURES. THE ACT OF LABELING SOMETHING WITH A WORD DISGUISES A CHALLENGABLE INDUCTIVE INFERENCE YOU ARE MAKING, BECAUSE WORDS ARE HIDDEN INFERENCES", BILLIAC said.
"Okay, so what?", pleaded Nick.
"WHAT PRIOR ASSUMPTIONS DOES THE LABEL 'UTILITY FUNCTION' SMUGGLE IN?"
At that moment, Nick was enlightened.
The Chinese stated preferences here closely track Western revealed preferences. Americans are more likely to dismiss AI risk post-hoc in order to justify making more money, whereas it seems that Chinese people are less likely to sacrifice their epistemic integrity in order to feel like a Good Guy, Hire people, and pay them money!