How do you know a mind's preference ordering? If this can't be taken for granted, then you use some of your evidence to infer the mind's preference ordering, and then use the inferred preferences to infer the mind's power, then use those two beliefs to testably predict future outcomes. Or you can use the Minimum Message Length formulation of Occam's Razor: if you send me a message telling me what a mind wants and how powerful it is, then this should enable you to compress your description of future events and observations, so that the total message is shorter. Otherwise there is no predictive benefit to viewing a system as an optimization process.
I liked this construction and now I want to see how well I can describe the power and wants of my college friends in order to make accurate predictions about future events and optimization. Because if I can't, I should work a little harder at staying in touch.
Bonus problem: do you know your own preference ordering in the sense above?
"This process prefers to exactly follow the laws of physics, therefore future events and observations will turn out exactly as a natural physical system would evolve" seems to be a minimal message length description for predicting the behavior of any process, unless that process necessarily restricts its final output to a very tiny domain that's shorter to describe than the initial state. For any description of the current measurable state of a process and a predicted future description of state likelihoods it seems that it will always be simpl...
Today's post, Optimization was originally published on 13 September 2008. A summary (taken from the LW wiki):
Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Psychic Powers, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.