Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Constant2 comments on Semantic Stopsigns - Less Wrong

54 Post author: Eliezer_Yudkowsky 24 August 2007 07:29PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (103)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Constant2 25 August 2007 12:09:53AM 10 points [-]

Synonyms for "I don't know".

There might be more of these than we think. Two candidates:

Aesthetics. People have a lot of understandable preferences: they prefer bigger houses to smaller ones, longer vacations to shorter ones, air conditioned rooms to hot and humid ones, and so on. What these have in common is that we can easily understand and explain the preferences. Aesthetic preferences, however, are generally characterized by it being hard, maybe impossible, to explain what it is about one thing that makes it more aesthetically pleasing than another. This suggests when we say "aesthetically pleasing", we almost mean, "pleasing, but if you asked why I wouldn't really be able to give you a satisfactory explanation."

Intelligence. Many have observed that each new success in reproducing, in machinery, the capabilities of the human mind, has in turn led to a narrowing of what is considered "intelligent" - a narrowing that excludes whatever it is that machines can now do. This phenomenon is explained if "intelligent behavior" is a subclass of some larger class of behavior (a larger class that includes the things that we have gotten machines to do, such as play a strong game of chess). In particular, the subclass labeled "intelligent" is that subclass of behaviors whose mechanism we have not yet discovered. I have often heard or read people say something like, "this is not intelligent behavior because what the computer is doing is [description of mechanism]." What really seems to be getting said is, "this is not intelligent behavior because I am able to describe it to you."