See also: Evolutionary Psychology, Goodhart'Goodhart's Law, Wireheading
A candy bar is a superstimulus: it contains more concentrated sugar, salt, and fat than anything that exists in the ancestral environment.
A candy bar matches taste buds that evolved in a hunter-gatherer environment, but it matches those taste buds much more strongly than anything that actually existed in the hunter-gatherer environment. The signal that once reliably correlated to healthy food has been hijacked, blotted out with a point in tastespace thatwasn'wasn't in the training dataset - an impossibly distant outlier on the old ancestral graphs.
See also: Evolutionary Psychology, Goodhart's Law, Wireheading
See also: Evolutionary Psychology, Goodhart's Law
A candy bar is a superstimulus: it contains more concentrated sugar, salt, and fat than anything that exists in the ancestral environment. A candy bar matches taste buds that evolved in a hunter-gatherer environment, but it matches those taste buds much more strongly than anything that actually existed in the hunter-gatherer environment. The signal that once reliably correlated to healthy food has been hijacked, blotted out with a point in tastespace that wasn't in the training dataset - an impossibly distant outlier on the old ancestral graphs.
-- Eliezer Yudkowsky, Superstimuli and the Collapse of Western Civilisation
Humans evolved various desires that promoted survival and reproductive success in the ancestral
environment.environment. Superstimuli are modern inventions that satisfy desires better than anything in the ancestral environmentcould,could but are detrimental to survival, reproduction, or otherhighhigh-level goals.