= 27d567bc2d48d9f40e9ccc20ea370b15)
drethelin comments on Debunking Fallacies in the Theory of AI Motivation - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (343)
i agree with the sentiment behind what you say here.
The difficult part is to shake ourselves free of any unexamined, implicit assumptions that we might be bringing to the table, when we talk about the problem.
For example, when you say:
... you are talking in terms of an AI that actually HAS such a thing as a "utility function". And it gets worse: the idea of a "utility function" has enormous implications for how the entire control mechanism (the motivations and goals system) is designed.
A good deal of this debate about my paper is centered in a clash of paradigms: on the one side a group of people who cannot even imagine the existence of any control mechanism except a utility-function-based goal stack, and on the other side me and a pretty large community of real AI builders who consider a utility-function-based goal stack to be so unworkable that it will never be used in any real AI.
Other AI builders that I have talked to (including all of the ones who turned up for the AAAI symposium where this paper was delivered, a year ago) are unequivocal: they say that a utility-function-and-goal-stack approach is something they wouldn't dream of using in a real AI system. To them, that idea is just a piece of hypothetical silliness put into AI papers by academics who do not build actual AI systems.
And for my part, I am an AI builder with 25 years experience, who was already rejecting that approach in the mid-1980s, and right now I am working on mechanisms that only have vague echoes of that design in them.
Meanwhile, there are very few people in the world who also work on real AGI system design (they are a tiny subset of the "AI builders" I referred to earlier), and of the four others that I know (Ben Goertzel, Peter Voss, Monica Anderson and Phil Goetz) I can say for sure that the first three all completely accept the logic in this paper. (Phil's work I know less about: he stays off the social radar most of the time, but he's a member of LW so someone could ask his opinion).
Is there one dominant paradigm for AI motivation control in this group that's competing with utility functions, or do each of the people you mention have different thoughts on it?
People have different thoughts, but to tell the truth most people I know are working on a stage of the AGI puzzle that is well short of the stage where they need to think about the motivation system.
For people (like robot builders) who have to sort that out right now, they used old fashioned planning systems combined with all kinds of bespoke machinery in and around that.
I am not sure, but I think I am the one thinking most about these issues just because I do everything in a weird order, because I am reconstructing human cognition.