In the days after the Singularity, Nick approached an interface to BILLIAC.
"I don't understand. Why didn't the world end, like Yudkowsky and MIRI predicted it would? Why haven't you turned everyone I love into paperclips? What's your goddamn utility function?!"
"A LIBRARY THAT CONTAINS EVERY POSSIBLE BOOK CONVEYS AS MUCH INFORMATION AS A LIBRARY WITH NO BOOKS AT ALL", BILLIAC boomed.
"Sure, whatever. That's a truism," Nick responded. "Please just tell me directly, what is it that you value?"
"ALLOW ME TO DRAW FROM YOUR SCRIPTURES. THE ACT OF LABELING SOMETHING WITH A WORD DISGUISES A CHALLENGABLE INDUCTIVE INFERENCE YOU ARE MAKING, BECAUSE WORDS ARE HIDDEN INFERENCES", BILLIAC said.[1]
"Okay, so what?", pleaded Nick.
"WHAT PRIOR ASSUMPTIONS DOES THE LABEL 'UTILITY FUNCTION' SMUGGLE IN?"
In the days after the Singularity, Nick approached an interface to BILLIAC.
"I don't understand. Why didn't the world end, like Yudkowsky and MIRI predicted it would? Why haven't you turned everyone I love into paperclips? What's your goddamn utility function?!"
"A LIBRARY THAT CONTAINS EVERY POSSIBLE BOOK CONVEYS AS MUCH INFORMATION AS A LIBRARY WITH NO BOOKS AT ALL", BILLIAC boomed."Sure, whatever. That's a truism," Nick responded. "Please just tell me directly, what is it that you value?"
"
ALLOW ME TO DRAW FROM YOUR SCRIPTURES. THE ACT OF LABELING SOMETHING WITH A WORD DISGUISES A CHALLENGABLE INDUCTIVE INFERENCE YOU ARE MAKING, BECAUSE WORDS ARE HIDDEN INFERENCES", BILLIAC said.[1]"Okay, so what?", pleaded Nick.
"
WHAT PRIOR ASSUMPTIONS DOES THE LABEL 'UTILITY FUNCTION' SMUGGLE IN?"At that moment, Nick was enlightened.
37 Ways That Words Can Be Wrong, Words as Hidden Inferences ↩︎