You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

buybuydandavis comments on My new paper: Concept learning for safe autonomous AI - Less Wrong Discussion

18 Post author: Kaj_Sotala 15 November 2014 07:17AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (16)
Tags: concept_learning ai friendly paper

Comments (16)

You are viewing a single comment's thread.

Comment author: buybuydandavis 16 November 2014 09:24:54PM 4 points [-]

I was immediately confused by the first two sentences in the abstract:

Sophisticated autonomous AI may need to base its behavior on fuzzy concepts that cannot be rigorously defined, such as well-being or rights. Obtaining desired AI behavior requires a way to accurately specify these concepts

We may need something that can't be done, but wait, we do require it, so I guess we better figure out how.

Are you making a distinction between defining and specifying?

If you just removed

that cannot be rigorously defined, the abstract reads perfectly sensibly and informatively.

I'm not sure what you're trying to add to the abstract with that phrase, but as is it mainly adds confusion for me.

Comment author: Toggle 17 November 2014 12:34:48AM *  4 points [-]

I read this as distinguishing between (on the one hand) an externally defined set of parameters and (on the other hand) a locally emergent pattern that may be too complex to be readily understood but which nonetheless produces behavior that conforms to our expectations for the concept. Consider Google's surprising 2012 discovery of cats.

You can teach somebody about the moon by describing it very precisely, or you can teach them about the moon by pointing to the moon and saying 'that thing.' In the latter case, you have specified a concept without defining it.

Comment author: buybuydandavis 17 November 2014 07:18:15AM 2 points [-]

That's a lot of meaning to be hanging on "defining" and "specifying".

Could entirely be what he meant. I guessed something similar, but I wouldn't want a reader having to guess at the meaning of an abstract.

Comment author: Kaj_Sotala 17 November 2014 09:41:19AM *  3 points [-]

Thanks for pointing that out, I didn't realize that the intended meaning was non-obvious! Toggle's interpretation is basically right: "rigorously defined" is referring to something like giving the system a set of necessary and sufficient criteria for when something should qualify as an instance of the concept. And "specifying" is intended to refer to something more general, such as building the system in such a way that it's capable of learning the concepts on its own, without needing an exhaustive (and impossible-to-produce) external definition of them. But now that you've pointed it out, it's quite true that the current choice of words doesn't really make that obvious: I'll clarify that for the final version of the paper.

Comment author: buybuydandavis 23 November 2014 02:00:01AM *  1 point [-]

Obtaining desired AI behavior

Looks like you're making distinctions between how you're going to build something that has the desired behavior. That how would be the specification.

These concepts could be explicitly specified set theoretically as concepts, or specified by defining boundaries in some conceptual space, or more generally, specified algorithmically as the product of information processing system with learning behavior and learning environment, without initially explicitly creating a conceptual representation.

It's not that one way is rigorous, and one is not, but that they are different ways of creating something with the desired behavior, or in your particular case, different ways between creating the concepts you want to use in producing the desired behavior. The distinction between a conceptual specification and an algorithmic specification seems meaningful and useful to me,

I think this works as a drop in replacement for the first two sentences:

Sophisticated autonomous AI may need to base its behavior on fuzzy concepts such as well-being or rights to obtain desired AI behavior. These concepts are notoriously difficult to explicitly define conceptually, but we explore implicitly defining those concepts by algorithmically generating those concepts.

I assumed that the type of AI design you're exploring is structurally committed to creating those concepts, instead of simply creating algorithms with the desired behavior, or I would have made more general statements about functionality.

Whatever you think of my proposed wording, and even if you don't like the distinctions I've made, the crucial word that I've added is but - an adversity conjuction. But, while, instead, ... a word to balance the things you're trying to make the distinction between, thereby identifying them. The meaning you intended in the first two sentences was a tension or conflict, but the grammar and sentence structure didn't reflect that.

Comment author: Kaj_Sotala 30 November 2014 09:48:53AM 0 points [-]

Thanks. I ended up going with:

Sophisticated autonomous AI may need to base its behavior on fuzzy concepts such as well-being or rights. These concepts cannot be given an explicit formal definition, but obtaining desired behavior still requires a way to instill the concepts in an AI system. To solve the problem, we review evidence suggesting that the human brain generates its concepts using a relatively limited set of rules and mechanisms. This suggests that it might be feasible to build AI systems that use similar criteria for generating their own concepts, and could thus learn similar concepts as humans do. Major challenges to this approach include the embodied nature of human thought, evolutionary vestiges in cognition, the social nature of concepts, and the need to compare conceptual representations between humans and AI systems.

Comment author: buybuydandavis 01 December 2014 04:09:38AM 1 point [-]

At least for me, this very clearly identifies the problem and your proposed approach to tackling it.

Powered by Reddit Powered by Reddit