Well, he might. Or, rather, there might be available ways of becoming smarter or prettier for which jettisoning his false beliefs is a necessary precondition.

But, admittedly, he might not.

Anyway, sure, if Joe "terminally" values his beliefs about the world, then he gets just as much utility out of operating within a VR simulation of his beliefs as out of operating in the world. Or more, if his beliefs turn out to be inconsistent with the world.

That said, I don't actually know anyone for whom this is true.

In this example, Joe's belief that he's smart and beautiful does pay rent in anticipated experience. He anticipates a favorable reaction if he approaches a girl with his gimmick and pickup line. As it happens, his innaccurate beliefs are paying rent in inaccurate anticipated experiences, and he goes wrong epistemically by not noticing that his actual experience differs from his anticipated experience and he should update his beliefs accordingly.

The virtue of making beliefs pay rent in anticipated experience protects you from forming incoherent beleifs, maps not corresponding to any territory. Joe's beliefs are coherent, correspond to a part of the territory, and are persistantly wrong.

But why do beliefs need to pay rent in anticipated experiences? Why can’t they pay rent in utility?

Is there a difference between utility and anticipated experiences? I can see a case that utility is probability of anticipated, desired experiences, but for purposes of this argument, I don't think that makes for an important difference.

But why do beliefs need to pay rent in anticipated experiences? Why can’t they pay rent in utility?

If some average Joe believes he’s smart and beautiful, and that gives him utility, is that necessarily a bad thing? Joe approaches a girl in a bar, dips his sweaty fingers in her iced drink, cracks a piece of ice in his teeth, pulls it out of his mouth, shoves it in her face for demonstration, and says, “Now that I’d broken the ice—”

She thinks: “What a butt-ugly idiot!” and gets the hell away from him.

Joe goes on happily believing that he’s smart and beautiful.

For myself, the answer is obvious: my beliefs are means to an end, not ends in themselves. They’re utility producers only insofar as they help me accomplish utility-producing operations. If I were to buy stock believing that its price would go up, I better hope my belief paid its rent in correct anticipation, or else it goes out the door.

But for Joe? If he has utility-pumping beliefs, then why not? It’s not like he would get any smarter or prettier by figuring out he’s been a butt-ugly idiot this whole time.

And the point that I am trying to get you to understand, is that you do not need special rule to always check P(2) when making a proof by induction, in this case where the induction fails at P(1) -> P(2), carefully trying to prove the induction step will cause you to realize this. More generally you cannot rigorously prove that for all integers n > 0, P(n) -> P(n+1) if it is not true, and in particular if P(1) does not imply P(2).

Leaving aside the actual argument, I can tell you that there exist people (my husband is one of them, and come to think of it so is my ex-girlfriend, which makes me suspect that I bear some responsibility here, but I digress) whose immediate emotional reaction to "here, let me walk you through this illustrative hypothetical case" is strongly negative.

The reasons given vary, and may well be confabulatory.

I've heard the position summarized as "I don't believe in hypothetical questions," which I mostly unpack to mean that they understand that hypothetical scenarios are often used to introduce assumptions which support conclusions that the speaker then tries to apply by analogy to the real world, and that a clever rhetoritician can use this technique to sneak illegitimate assumptions into real-world scenarios, and don't trust me not to sneak in assumptions that make them look stupid or manipulate them into acting against their own interests.

I don't know if that's a factor in your case or not, but I have found that once I trigger that reaction, there's not much more I can do... they are no longer cooperating in the communication, they are just looking for some way to get out. If I press the point, I merely elicit anger and defensiveness and a variety of distractors.

The best way around this I've found so far, and it's only hit-or-miss, is to avoid the stance of "here let me show you something" altogether.

I am a lot more successful if I adopt the stance of "I am thinking about a problem that interests me," and if they express interest, explaining the problem as something I am presenting to myself, rather than to them. Or, if they don't, talking about something else.

At the risk of sounding like Robin, the fact that this is successful leads me to believe that at least sometimes, what's really going on is that I've stepped on some status-signaling landmine, and the reaction I'm getting actually translates to "I refuse to cede you the role of instructor by letting you define the hypothetical."

And suggesting that this might be what's going on works about as poorly as you'd expect it to were it what's going on. Of course, that's precisely what makes status-signaling a fully generalizable counterargument, so I take it with a grain of salt.

The first n horses and the second n horses have an overlap of n-1 horses that are all the same color. So first and the last horse have to be the same color.

You need to make this more explicit, to expose the hidden assumption:

Take a horse from the overlap, which is the same color as the first horse and the same color as the last horse, so by transitivity, the first and last horse are the same color.

But why can you take a horse from the overlap? You can if the overlap is non-empty. Is the overlap non-empty? It has n-1 horses, so it is non-empty if n-1 > 0. Is n-1 > 0? It is if n > 1. Is n > 1? No, we want the proof to cover the case where n=1.

Suppose every n horses are of one color. Add the n+1st horse, and take n out of those horses. They’re all of one color by assumption. Remove 1 horse and take the one that’s been left out. You again have n horses, so they must be of one color. Therefore, all horses are of one color.

You didn't actually prove that n+1 horses have one color with this, you know, even given the assumption. You just said twice that n horses have one color, without proving that their combined set still has one color.

For example consider the following "Suppose every n horses can fit in my living room. Add the n+1 horse, and take n out of those horses. They can fit in my living room by assumption. Remove 1 horse and take the one that’s been left out. You again have n horses, so they must again fit in my living room. Therefore, all horses fit in my living room."

That's not proper induction. It doesn't matter if you begin with a n of 1, 2, 5, or 100 horses, such an attempt at induction would still be wrong, because it never shows that the proposition actually applies for the set of n+1.

Why didn't you drop the "horses" example when it tripped him up and go with, I dunno, emeralds or ceramic pie weights or spruckels, stipulated to in fact have uniform color?

Him: Since the result is wrong, the proof is wrong. Period. Stop wasting my time with this pointless stuff. This is stupid and pointless, etc, etc. Whoever teaches you this stuff should be fired.

...

What on Earth went wrong here?

The problem was that your ultimate conclusion was wrong. It is not in fact the case that "mathematical induction should start with the second step, not the first." It's just that, like all proofs, you have to draw valid inferences at each step. As JGWeissman points out, the horse proof fails at the n=2 step. But one could contrive examples in which the induction proof fails at the kth step for arbitrary k.