Today's post, Optimization was originally published on 13 September 2008. A summary (taken from the LW wiki):
A discussion of the concept of optimization.
Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Psychic Powers, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
An oil refinery is slightly intelligent as an optimization process. Presumably it has sensors that can detect the heaviness of the crude input, its sulfur content, and perhaps other impurities and process it accordingly so that valuable petroleum fractions are maximized. But I don't know that much about refineries; it may be the case that human operators madly turn valves and pull levers while watching spinning dials to make our gasoline and the refinery is just a big dumb tool.
The question to ask about the refinery as an optimization process is to what degree its initial conditions can be changed and it still produces optimized output. The more powerful it is as an optimizer, the more degrees of freedom on the input it can handle. A perfect refinery could accept waste from a landfill and always produce highly valuable goods.
If you're asking about future possible states, and literally counting them as Eliezer suggests, even a refinery that has very little input flexibility is a powerful optimizer, provided it puts out well-refined products. It's a more powerful optimizer than most people, by that metric. How do I know? It has influence over a lot more total mass. The total entropy reduction it performs is vast.
Why does it matter how humans are involved? That's one of the positive aspects of Eliezer's definition: it doesn't care about whether there are humans pulling levers, hu... (read more)