Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

SilasBarta comments on Understanding your understanding - Less Wrong

69 Post author: SilasBarta 22 March 2010 10:33PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (77)

You are viewing a single comment's thread. Show more comments above.

Comment author: SilasBarta 23 March 2010 04:12:45PM 1 point [-]

Actually, Stage 3 works as a standard for the scientific method as well. That is, if knowledge of that specific method were deleted from your mind, would you rediscover it? Do you have an epistemology that would come up with an idea like, "Hey, I need to check these general ideas I have against nature, to see if they really hold" without it having been revealed to you in advance?

Ideally, you'd come up with (or have to start from!) something even better: the Bayesian rationalist method, of which the scientific method is a crippled, special case. While science is better than superstition, it also permits slower updates than you can justify, and often allows certain kinds of evidence that you shouldn't count.

However, if you found yourself in a role analogous to "being one-eyed in the land of the blind", and others' minds weren't capable of following Bayesian rationality, then you may want to teach them the scientific method as a next-best epistemology.

Comment author: Johnicholas 23 March 2010 06:53:33PM 1 point [-]

What if "Bayesian rationality" were deleted?

How did "Bayesian rationality" get discovered, except by the usual practices of scientists? (I won't say "the scientific method", partly because it's really fuzzy and so the "the" at the beginning of the phrase is deceptively concrete, and partly because I don't think that the process is as tidy as descriptions of the scientific method make it out to be.)

If we're looking for an error-correcting system, we need to look for a vast number of weak epistemological principles, on the level of "if event X is followed almost immediately by event Y, guess that X is generally followed almost immediately by Y", along with perceptual details of "how long?" and "what should count as an event?".

They would probably be fiercely embodied, but that's not actually a problem - we're fiercely emphysicsed, after all.

Comment author: SilasBarta 23 March 2010 09:19:17PM 4 points [-]

Beyond a certain point, the "regenerate if deleted?" metric becomes useless. For example, if your entire source code is "0", well, everything's been deleted, but there's no way it's growing back. There has to be somewhere to start. (Related: Where recursive justification hits bottom)

Still, you can characterize epistemic states by how much they could recover, from how deep a deletion, which was one point of the Truly Part of You article. I can imagine simpler epistemic states, lacking knowledge of the scientific method, that could recover Bayesian rationality: you would need to recognize that primitive-future has dynamics very close to primitive-past (where primitive-X denotes the inborn, intuitive understanding of X), which gives you induction, and, combined with basic numeracy, could point you in the right direction.

Comment author: reaver121 24 March 2010 08:26:09AM 1 point [-]

That was my main problem with the definition of stage 3 and was why I posted my original comment. It seemed to me that you could apply stage 3 to parts of your knowledge but not for everything.

When I read 'This stage should be the goal of all rationalists.' (in the original post) I was confused because it seemed to me that stage 3 was unreachable. I mean, if I started with only my human psychology, my senses and the world around me (i.e. the level of a caveman) I don't think I would invent math, physics,... Stage 3 seemed reachable if I assumed infinite time & persistence and scientific reasoning.

Comment author: cupholder 23 March 2010 10:41:08PM *  1 point [-]

I don't know about deducing the entire mindset & toolbox of 'Bayesian rationality,' but knowing Bayes' theorem is the key part of it, and I wouldn't expect that to be too hard to reconstruct if you knew what you look to for.

Bayes' theorem follows trivially from the definition of conditional probability, and that definition is itself quite intuitive. So in theory, once you have a feel for what probability is, it'd be quite possible to get to Bayes' theorem. I haven't read Huygens' 1657 book on probability theory, but if it was any good, I bet Huygens knew enough about it to beat Bayes to Bayes' theorem by a century.

Comment author: mattnewport 23 March 2010 11:12:51PM *  2 points [-]

Chapters 1 and 2 of Jaynes' Probability Theory: The Logic of Science show how Bayes' theorem follows necessarily from certain basic principles of plausible reasoning. In some sense all roads lead to Bayes when trying to derive a consistent mathematical procedure for manipulating degrees of plausibility.

Comment author: cupholder 23 March 2010 11:25:00PM 0 points [-]

You are quite right. I thought about mentioning the Cox-Jaynes road to Bayes' theorem in my post, but decided that someone trying to reconstruct Bayes' theorem would be more likely to get to it by muddling through intuitively via conditional probability.