Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

sketerpot comments on Rationality quotes: August 2010 - Less Wrong

5 Post author: Cyan 03 August 2010 12:16AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (201)

You are viewing a single comment's thread.

Comment author: sketerpot 04 August 2010 07:10:17PM 17 points [-]

If you do experiments and you're always right, then you aren't getting enough information out of those experiments. You want your experiment to be like the flip of a coin: You have no idea if it is going to come up heads or tails. You want to not know what the results are going to be.

-- Peter Norvig, in an interview about being wrong. When I saw this, I thought it sounded a lot like entropy pruning in decision trees, where you don't even bother asking questions that won't make you update your probability estimates significantly. Then I remembered that Norvig was the co-author of the AI textbook that I had learned about decision trees from. Interesting interview.

Comment author: SilasBarta 04 August 2010 07:46:53PM 4 points [-]

Wow, I'm glad this kind of analysis is showing up in mainstream publications.

Norvig is describing an important insight from information theory: the amount of information you get from learning something is equal to the log of the inverse of the probability you assigned to it (log 1/p). (This value is called the "surprisal" or "self-information".)

So, always getting results you expect (i.e. put a high p on), means you're getting little information out of the experiments, and you should be doing ones where you expect the result to be less probable.

Therefore, to have a good experiment, you want to maximize the "expected surprisal" (i.e. sum over p * log(1/p)), which is equivalent to the entropy, and probably the basis for the method you mention.

Comment author: Oscar_Cunningham 04 August 2010 08:32:19PM *  1 point [-]

Is LW broken for everyone?

ETA: When I wrote this, the "Comments" page was one of the few I could access, hence it being posted in such a strange place.

Comment author: RichardKennaway 04 August 2010 09:26:07PM 5 points [-]

No, only for the ones who can't reply to your comment.

Comment author: NancyLebovitz 04 August 2010 08:49:13PM 0 points [-]

It's broken for me, at least.

Instead of the old-fashioned way of breaking, with new comments around a certain time not being visible, I can't get to older comments or posts. I don't know what the extent of the loss of access is.

The report issues page (link at bottom of page) is read only. There was going to be a :"brief outage" for that page at 7AM PDT. That would be almost 7 hours ago.

I think the stock of humorous messages at the "you tried a link which isn't working" has been improved.

Comment author: NancyLebovitz 04 August 2010 09:00:56PM 0 points [-]

The last line should have been

I think the stock of humorous messages at the "you tried a link which isn't working" page has been improved.

Or to put it another way, the permalink which would normally make it possible for me to edit isn't working either.

Comment author: NancyLebovitz 04 August 2010 09:03:25PM 0 points [-]

And, though this is minor, if I hit the comment button, it seems as though nothing has happened. However, if I refresh the comments page, my comment shows up.

As a minor mercy, hitting the comment button more than once doesn't seem to produce duplicate comments.

Comment author: cupholder 04 August 2010 07:20:53PM 0 points [-]