Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

TheAncientGeek comments on Could utility functions be for narrow AI only, and downright antithetical to AGI? - Less Wrong Discussion

5 Post author: chaosmage 16 March 2017 06:24PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (38)

You are viewing a single comment's thread. Show more comments above.

Comment author: TheAncientGeek 20 March 2017 10:13:34PM 0 points [-]

You could model humans as having varying UFs, or having multiple UFs...or you could give up on the whole idea.

Comment author: denimalpaca 21 March 2017 06:46:20PM 0 points [-]

Why would I give up the whole idea? I think you're correct in that you could model a human with multiple, varying UFs. Is there another way you know of to guide an intelligence toward a goal?

Comment author: TheAncientGeek 22 March 2017 02:57:47PM 2 points [-]

The basic problem is the endemic confusion between the map, the UF as a way of modelling an entity, and the territory. the UF as an architectural feature that makes certain things happen.

The fact that there are multiple ways of modelling humans as UF-driven, and the fact that they are all a bit contrived, should be a hint that there may be no territory corresponding to the map.

Comment author: denimalpaca 22 March 2017 06:00:08PM 0 points [-]

Is there an article that presents multiple models of UF-driven humans and demonstrates that what you criticize as contrived actually shows there is no territory to correspond to the map? Right now your statement doesn't have enough detail for me to be convinced that UF-driven humans are a bad model.

And you didn't answer my question: is there another way, besides UFs, to guide an agent towards a goal? It seems to me that the idea of moving toward a goal implies a utility function, be it hunger or human programmed.

Comment author: TheAncientGeek 24 March 2017 12:35:09PM *  0 points [-]

Is there an article that presents multiple models of UF-driven humans and demonstrates that what you criticize as contrived actually shows there is no territory to correspond to the map?

Rather than trying to prove the negative, it is more a question of whether these models are known to be useful.

The idea of mulitple or changing UFs suffers from a problem falsifiability, as well. Whenever a human changes their apparent goals, that's a switch to another UF, or a change in UF? Reminiscent of ptolemaic epicycles, as Ben Goerzel says.

And you didn't answer my question: is there another way, besides UFs, to guide an agent towards a goal? It seems to me that the idea of moving toward a goal implies a utility function, be it hunger or human programmed.

Implies what kind of UF?

If you are arguing tautologously that having a UF just is having goal directed behaviour, then you are not going to be able to draw interesting conclusions. If you are going to define "having a UF broadly, then you are going to have similar problems, and in particular the problem that "the problem of making an AI safe simplifies to the problem of making its UF safe" only works for certain, relatively narrow, definitions of UF. In the context of a biological organism, or an artificial neural net or deep learning AI, the only thing "UF" could mean is some aspect of its functioning that is entangled with all the others. Neither a biological organism, nor an artificial neural net or deep learning AI is going to have a UF that can be conveniently separated out and reprogrammed. That definition of UF only belongs in the context of GOFAI or symbolic programming.

There is no point in defining a term broadly to make one claim come out true, if it was is only an intermediate step towards some other claim, which doesn't come out as true under the broad definition.

Comment author: denimalpaca 24 March 2017 08:10:02PM 0 points [-]

My definition of utility function is one commonly used in AI. It is a mapping of states to a real number: u:E -> R where u is a state in E (the set of all possible states), and R is the reals in one dimension.

What definition are you using? I don't think we can have a productive conversation until we both understand each other's definitions.

Comment author: TheAncientGeek 25 March 2017 11:38:06AM 0 points [-]

I'm not using a definition, I'm pointing out that standard arguments about UFs depend on ambiguities.

Your definition is abstract and doens't capture anything that an actual AI could "have" -- for one thing, you can't compute the reals. It also fails to capture what UF's are "for".

Comment author: denimalpaca 25 March 2017 05:52:06PM 0 points [-]

Go read a textbook on AI. You clearly do not understand utility functions.

Comment author: TheAncientGeek 26 March 2017 08:32:47AM *  0 points [-]

AI researchers, a group of people who are fairly disjoint from LessWrongians, may have a rigorous and stable definition of UF, but that is not relevant. the point is that writings on MIRI and LessWrong use,and in fact depend on, shifting an ambiguous definitions.