Eliezer_Yudkowsky comments on Mutual Information, and Density in Thingspace - Less Wrong

20 Post author: Eliezer_Yudkowsky 23 February 2008 07:14PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (24)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Eliezer_Yudkowsky 24 February 2008 05:02:40PM 0 points [-]

Will, thingspace may not need a distance metric depending on how you draw your boundaries, which are not necessarily surfaces containing volumes of constant density. For example, a class in Naive Bayes / neural network of type 2 also slices up thingspace. More about this shortly. But if you're interested in the general topic, I believe that in the field of statistical learning, for algorithms that actually do depend on distance metrics, the standard cheap trick is to "sphere" the space by making the standard deviation equal 1 in all directions. An ad-hoc technique but apparently a useful one, though it has all the flaws you would expect.

tcpkac, see Kenneway's response.