Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

orthonormal comments on "What Is Wrong With Our Thoughts" - Less Wrong

23 Post author: Eliezer_Yudkowsky 17 May 2009 07:24AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (103)

You are viewing a single comment's thread. Show more comments above.

Comment author: orthonormal 18 May 2009 06:29:42PM 4 points [-]

The trouble there is the claim of a rational mind, in my opinion. It's not logically necessary that our evolved brains, hacked by culture, are going to mirror reality in their most basic perceptions and intuitions.

The space of all possible minds includes some which have a notion of number and counting and an intuitive mental arithmetic, but for which 2 and 2 really do seem to make 3 when they think of it. These minds, of course, would notice empirical contradictions everywhere: they would put two objects together with two more, count them, and count four instead of three, when it's obvious by visualizing in their heads that two and two make three instead. Eventually, a sufficiently reflective mind of this type would entertain the possibility that maybe two and two do actually make four, and that its system of visualization and mental arithmetic are in fact wrong, as obvious as they seem from the inside. Switching "three" and "four" in this paragraph just illustrates how difficult accepting that hypothesis might actually be for such a mind.

The thing is, we ourselves are in this situation, not with arithmetic (fortunately, we receive constant empirical reinforcement that 2+2=4 and that our mental faculties for arithmetic work properly) but with our biases of thought. Things like our preferences and valuations seem to be rational and coherent, in that we can usually defend them all with arguments that look solid and persuasive to us. But occasionally this fiction becomes untenable, as when we are shown to have circular preferences in situations of risk and reward. As Eliezer put it in Zut Allais:

You want to scream, "Just give up already! Intuition isn't always right!"

Or, in this case, "Don't start by assuming that our minds work rationally whenever they see something as obvious! If this is true, it is an empirical fact; and you should be able to see the alternative as possible!"

Comment author: byrnema 18 May 2009 09:21:32PM *  0 points [-]

If this is true, it is an empirical fact; and you should be able to see the alternative as possible!

Indeed, 2+2=4 is only true in some contexts. For example, sometimes 1+1=1 -- in contexts where separate objects lose their distinct identity as soon as they are grouped. (Think of a particular object several times. How many times did you think of it? But how many objects did you think of?)

Later edit: It is interesting that such a benign comment would get 4 down votes. Perhaps I understand this group well enough to guess why: the experiment I suggested is an entirely "internal" one, it provides no external proof of what I am suggesting. I think that a common reader here feels dismissive of, if not entirely antagonistically towards, knowledge that is internally generated. Personally, I have a preference for the knowledge that arises from internal experience.

Comment author: komponisto 19 May 2009 03:46:46AM 6 points [-]

I agree that the downvoting of this comment was overly harsh. My theory on why it occurred is different, and best illustrated by an example: if someone posted a comment saying "2+2=4 is only true in some contexts; in arithmetic modulo 3, 2+2=1", that comment would have been similarly downvoted.

However, let me be so bold as to say a word in defense of even that hypothetical commenter. Anyone mathematically sophisticated (including our downvoters) will agree that it is possible to construct a mathematical system in which 2+2 equals anything you like -- or, more precisely, for any symbol x, a system can be constructed in which the formula (string of symbols) "2+2 = x" is given the label "TRUE". Mod 3 arithmetic is an example for x = "1".

Now, it is at this point that the downvoters protest: "But this is not the same thing as saying 2+2=1! All you've done is change the meaning of the symbols in the formula, such as '2' and '1'. Two plus two is still four, for the original meaning of those words. You're confusing the map and the territory. Downvoted!"

Well, the downvoters do have a point. But, at the same time, let me suggest that they're also making the same mistake as our poor beleaguered commenter!

What they've done, you see, is to make a leap from "Ordinary (i.e. non mod-3, etc.) Arithmetic accurately models certain physical phenomena" to something like "Ordinary Arithmetic is true in (or of) the physical world". Instead of saying what they mean, which is "the physical world is best modeled by a system that has '2+2=4' as a 'TRUE' formula", they say "2+2 is in fact equal to 4".

Small wonder that confusion arises about whether mathematical statements are "emprical" or not! "The physical world is best modeled by a system that has '2+2=4' as a 'TRUE' formula" is clearly an empirical claim. But what about 2+2 = 4, all by itself? When a mathematician at a blackboard proves that 2+2=4 in Ordinary Arithmetic (or, for Eliezer's benefit, that infinite sets exist in standard set theory), has he or she made a claim about physics? No! Not without the additional assumption that the formal system being used is in fact an accurate map of the territory! But the mathematician makes no such assumption; he or she (acting as a mathematician) is interested only in the properties of formal systems. (Yes, that's right: I'm advocating the view known as formalism here. The other well-known positions in the philosophy of mathematics, namely Platonism and intuitionism, suffer from map-territory confusion!)

Mathematical systems, like Ordinary Arithmetic or Mod-3 Arithmetic, are part of the map, not the territory. The facts of mathematics are, so to speak, cartographic, rather than geographic.

Comment author: byrnema 19 May 2009 05:49:47AM *  1 point [-]

In the OB post tautologies have to be empirically observed somehow, Eliezer writes about waking up one day and discovering all sorts of evidence that 2+2=3. This wouldn't be evidence that 2+2=3 in Peano arithmetic, it would be evidence that Peano arithmetic just doesn't apply for some reason. In my down-voted comment, I was just giving an example of how there can be different kinds of arithmetic if you are willing to be flexible about what arithmetic is. (If you are not willing to be flexible, then you are not willing to allow the observation that 2+2=3 as an observation about arithmetic, because this is not possibly true in standard arithmetic. Well, the observations are possible but you'd have to account for it as some kind of grand delusion.) My point is that 2+2=4 in Peano Arithmetic independent of observation, but observation tells you if Peano arithmetic applies or not.

Comment author: komponisto 19 May 2009 07:30:00AM *  3 points [-]

This wouldn't be evidence that 2+2=3 in Peano arithmetic, it would be evidence that Peano arithmetic just doesn't apply for some reason.

Exactly.

My point is that 2+2=4 in Peano Arithmetic independent of observation, but observation tells you if Peano arithmetic applies or not.

It is worth emphasizing that to claim that "2+2=4 in Peano Arithmetic independent of observation" is not to claim that our knowledge of this fact about Peano Arithmetic is independent of observation. (The former claim is about our map of the territory; the latter is about our map of our map of the territory.)

Comment author: Alicorn 18 May 2009 09:56:14PM 1 point [-]

How many words are in this list?

  • Duck
  • Duck
  • Goose
Comment author: JGWeissman 18 May 2009 10:07:38PM 0 points [-]

The list contains 3 word instances and represents 2 word prototypes.

Comment author: Alicorn 18 May 2009 11:49:28PM 1 point [-]

The jargon I was looking for was 3 tokens of 2 types, but close enough :)

Comment author: JGWeissman 19 May 2009 07:36:32PM 0 points [-]

Heh, with the right jargon, you can accomplish anything. ;)

Comment author: steven0461 18 May 2009 09:28:17PM 1 point [-]

Doesn't that just mean that grouping doesn't always correspond to addition?

Comment author: byrnema 18 May 2009 11:09:39PM 1 point [-]

Yes. If 1+1 is anything other than 2, then it's not addition. I gave an example of how combining two things together doesn't always yield addition.