I do not think your claim is what you think it is.

I think your claim is that some people mistake the model for the reality, the map for the territory. Of course models are simpler than reality! That's why they're called "models."

Physics seems to have gotten wiser about this. The Newtonians, and later the Copenhagenites, did fall quite hard for this trap (though the Newtonians can be forgiven to some degree!). More recently, however, the undisputed champion physical model, whose predictions hold to 987 digits of accuracy (not really), has the humble name "The Standard Model," and it's clear that no one thinks it's the ultimate true nature of reality.

Can you give specific examples of people making big mistakes from map/territory confusion? The closest thing I can think of offhand is the Stern Report, which tries to make economic calculations a century from now based on our current best climate+social+political+economic models.

This post confused me enormously. I thought I must be missing something, but reading over the comments, this seems to be true for virtually all readers.

What exactly do you mean by "bead jar guess"? "Surprise"? "Actual probability"? Are you making a new point or explaining something existing? Are you purposely being obscure "to make us think"?

I propose replacing this entire post with the following text:

Hey everybody! Read E.T. Jaynes's Probability Theory: The Logic Of Science!

I strongly agree. As an anecdotal data point, I understood the suggested alternative but not the original wording. And it is a powerful point to miss because I haven't heard of Kahnemann and Tversky.

Also, if mentioning specific researchers were central to the point, I would recommend linking to a resource about them, or better yet, create entries for them on the Less Wrong Wiki and link to those.

Take a second to go upvote You Are A Brain if you haven't already...

This is extremely off-topic, but please do not tell me what to upvote. I actually downvoted that post because the slideshow was completely useless to me and I thought its quality was poor. This isn't to slam Liron; his post just didn't do it for me.

But just because you really, really liked it doesn't mean you get to tell me what to like.

Seconded! Those names didn't ring a bell for me either, though I'm familiar with the results from Prespect Theory (I probably read about them on OB), and that's probably what talisman was refering to.

For instance, reasoning by expected utility, which you probably consider too basic to mention

Actually, I consider it too complicated for my first book! That's going to focus on getting across even more basic concepts like 'the point of reasoning about your beliefs is to function as a mapping engine that produces correlations between a map and the territory' and 'strong evidence is the sort of evidence we couldn't possibly find if the hypothesis were false'.

Upvoted for calling your own post "completely wrong"!

While I agree in this case, your criticism as stated is so vague that it could be made of any idea

Yet is wasn't made of any idea, and if you agree in this case, you deem it appropriate for this case, in which it was made. It is a fairly general principle: don't try to juggle a thousand angry cats at once, human mind is not that strong.

• Let me add to the chorus of "you rock!" This is a nice piece of work. I don't know how you got the chance to present to a group of young people about this stuff, but kudos also to whoever gave you that opportunity.
• Some have pointed out potential improvements. This seems like a solid way for anyone interested to add a quantum of effort to the cause---improve the presentation a bit, and post your improved version somewhere. (Where?)

I'm someone who has been educated in CS, as well as other fields. I both perform and manage industrial-scale programming and research programming, and I hire programmers of varying skill-levels for projects.

In my experience, there are a few caveats to your advice:

1) Strong programmers, fun side projects, and deep understanding comes from the pre-Computer interest which many of the die-hard CS people have. It is rarely, if ever, taught in academic programs and is something I specifically look for in my heavy-duty programmers whether or not their major is or was in CS.

2) My personal take on things is: The best programmers take innate talent and a lifetime of experience and turn it into super-star ability - don't expect to become one by majoring in CS. The second best programmers have a strong mathematics background (i.e. proof writing), because their code requires less debugging, which is the most costly part of software development. Any programmer can do better by analyzing their process and learning from that, but that form of software engineering seems to have become less popular, since it takes more work and makes you vulnerable to bad managers. Algorithm design is, of course, an entirely different matter.

In short, I'd suggest a CS/Mathematics double major. It's a good idea to be able to speak more than one technical language after all.

Oh and, my usual answer to:

When's the last time you heard a math person refer to some real-world situation as "a real elliptic curve"?

Well there was the time Heisenberg came back from his honeymoon where he had, of course, spent the entire time trying to puzzle out quantum mechanics. He was about to give up because of the non-commutative multiplication when he talked to a mathematician friend who said, "Wait, let me tell you about these things we call matrices."

Mathematics gives you very useful models, it just doesn't tell you how to use them. CS gives you the tools to implement solutions, but it tends to leave the solutions rather ad hoc. Bringing the two together is one good mix. I could suggest a half-dozen others - but this will do for now.

Edited to add: This is a mix if you want to achieve the goals laid out in the post. Not suggesting that it is either trivial to get such a pair of degrees or desirable for everyone to head in that direction.