timtyler comments on Why the beliefs/values dichotomy? - Less Wrong

20 Post author: Wei_Dai 20 October 2009 04:35PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (153)

You are viewing a single comment's thread. Show more comments above.

Comment author: RichardKennaway 21 October 2009 04:53:29PM *  2 points [-]

You seem engaged in pointless hair-splitting. The Roomba's designers wanted it to clean floors. It does clean floors. That is what it is for. That is its aim, its goal.

To me, the distinction between a purposive machine's own purposes, and the purposes of its designers and users is something that it is esssential to be clear about. It is very like the distinction between fitness-maximising and adaptation-executing.

As for me having to exhibit a whole model of a Roomba to illustrate that such a model could be built - that is crazy talk. You might as well argue that I have to exhibit a model of a suspension bridge to illustrate that such a model could be built.

As a matter of fact, you would have to do just that (or build an actual one), had suspension bridges not already been built, and having already well-known principles of operation, allowing us to stand on the shoulders of those who first worked out the design. That is, you would have to show that the scheme of suspending the deck by hangers from cables strung between towers would actually do the job. Typically, using one of these when it comes to the point of working out an actual design and predicting how it will respond to stresses.

If you're not actually going to build it then a BOTE calculation may be enough to prove the concept. But there must be a technical explanation or it's just armchair verbalising.

The utility maximiser framework can model the actions of any computable intelligent agent - including a Roomba. That is, so long as the utility function may be expressed in a Turing-complete language.

If this is a summary of something well-known, please point me to a web link. I am familiar with stuff like this and see there no basis for this sweeping claim. The word "intelligent" in the above also needs clarifying.

What is a Roomba's utility function? Or if a Roomba is too complicated, what is a room thermostat's utility function? Or is that an unintelligent agent and therefore outside the scope of your claim?

Comment author: timtyler 21 October 2009 06:52:07PM 1 point [-]

By all means distingush between a machine's purpose, and that which its makers intended for it.

Those ideas are linked, though. Designers want to give the intended purpose of intelligent machines to the machines themselves - so that they do what they were intended to.