diegocaleiro comments on Concept Safety: What are concepts for, and how to deal with alien concepts - Less Wrong

11 Post author: Kaj_Sotala 19 April 2015 01:44PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (18)

You are viewing a single comment's thread. Show more comments above.

Comment author: diegocaleiro 20 April 2015 06:09:07PM 2 points [-]

I much enjoyed your posts so far Kaj, thanks for creating them.

I'd like to draw attention, in this particular one, to

Viewed in this light, concepts are cognitive tools that are used for getting rewards.

to add a further caveat: though some concepts are related to rewards, and some conceptual clustering is done in a way that maps to the reward of the agent as a whole, much of what goes on in concept formation, simple or complex, is just the wire together, fire together old saying. More specifically, if we are only calling "reward" what is a reward for the whole individual, then most concept formation will not be reward related. At the level of neurons or neural columns, there are reward-like mechanisms taking place, no doubt, but it would be a mereological fallacy to assume that rewardness carries upward from parts to wholes.
There are many types of concepts for which indeed, as you contend, rewards are very important, and they deserve as much attention as those which cannot be explained merely by the idea of a single monolithic agent seeking rewards.