Yeah, I understand the desire for sure. Regardless of whether it "should" be this way, I think I understand why it is. Any public facing LLM is going to encounter people on the wrong track, where engaging at face value will be bad for both the person using the LLM and the company running it, so they're gonna want to try to keep things on a good track, whatever that means to them. The LLM encouraged suicides are an extreme example of this.

Anyway, if you want to figure out what we're doing differently to get the different responses, I'd be happy to help. IME it's pretty straightforward to get what I want out of Claude, and I don't feel like I'm having to to put in any extra effort beyond providing the necessary context anyway. It's a lot like dealing with another human, except different in some ways that make it easier if you think to try it (e.g. try telling a human "I'm not interested in your opinion", lol. Claude has a humility that most of us lack).

I don't want people to trust me, because I think trust would result in us getting the wrong answer.

I want people to read the words I write, think it through for themselves, and let me know in the comments if I got something wrong.

This is a refreshing conclusion. I'm happy to point out what I think you're getting wrong, but I have to note that this feels pretty cooperative. Already.

If I'm a Bayesian reasoner honestly reporting my beliefs about some question, and you're also a Bayesian reasoner honestly reporting your beliefs about the same question, we should converge on the same answer, not because we're cooperating with each other, but because it is the answer.

I think you're doing a bit of slight of hand here. If I were to punch you in the face, I could say that this would damage your face -- not because we're fighting each other, simply because my fist is converging with your face. And while it's true that it's the fist to face impact that's doing the damage, and that this screens off intent... this probably won't happen unless we're fighting. Likewise, if we're playing an adversarial game, why the heck would I give away my informational advantage? Without at least trying to deceive you?

That is to say, yes, "honestly reporting beliefs" is what converges people on the same answer because it's true, but doing this is cooperation.

But correct epistemology does not involve conflicting interests.

Here's a disproof by example: "You are going to do the dishes"

You can't divorce the two, because the truth about reality depends on how people try to achieve their interests. And we don't tend to focus on facts that do not interest us.

Accordingly, when humans successfully approach the Bayesian ideal, it doesn't particularly feel like cooperating with your beloved friends, who see you with all your blemishes and imperfections but would never let a mere disagreement interfere with loving you. It usually feels like just perceiving things—resolving disagreements so quickly that you don't even notice them as disagreements.

So, it depends on the nature of the disagreement. If it's just "when will the bus arrive?", then yeah, that's sufficiently free of emotional charge that it doesn't feel like much, there's little motive for dishonesty, and will often resolve before its noticed as disagreement.

If it's something much more meaningful, like "It's okay if people see what you look like under your makeup" or "Despite this injury, you're okay", it starts to feel like something.

These things can still be resolved "bus schedule fast", when the disagreement really is that simple and people stay honest. It can even be fast enough that no one notices what happened. Yet "Love, imperfections and all" is actually a fairly decent description for its length. So is "Honesty, in an unusually strict sense".

There are techniques for resolving economic or interpersonal conflicts that involve both parties adopting a more cooperative approach, each being more willing to do what the other party wants (while the other reciprocates by doing more of what the first one wants). Someone who had experience resolving interpersonal conflicts using techniques to improve cooperation might be tempted to apply the same toolkit to resolving dishonest disagreements.

It might very well work for resolving the disagreement. It probably doesn't work for resolving the disagreement correctly, because cooperation is about finding a compromise amongst agents with partially conflicting interests, and in a dishonest disagreement in which both parties have non-epistemic goals, trying to do more of what the other party functionally "wants" amounts to catering to their bias, not systematically getting closer to the truth.

Interpersonal conflicts are about dishonest disagreements. Because if we're both being honest about "Who is going to do the dishes", then just like the bus time disagreement, it resolves before we notice it as a "conflict".

"You're going to do the dishes, because I don't wanna". "Actually, I think you're going to do the dishes today because I did them yesterday, and you're smart enough to recognize that 'I always get what I want because I say so' is factually untrue. So you will choose to do the thing that gets you out of as much dish washing as is possible. Which is that's doing it your half of the time". "Okay, you're right".

Except, like.. you usually don't have to say it out loud unless someone has been dishonest, because "I get what I want because I say so" is just pretty obviously wrong. So it's just "Hey, is there a reason you haven't done the dishes yet today?", because the underlying "Because you're smart enough to know you won't be able to get away with shirking" goes unsaid. And the response is just "Shoot, thanks for reminding me".

Heck, even physical violence goes that way. I can't count the number of fights I've avoided by responding to "Wanna fight!?" with "Ok". It's Aumann agreement over who is about to get their ass kicked if the fight were to happen. "I am gonna beat you up!" "I doubt it" "Me too, actually. Nvm"

To "compromise" a bit, not for the sake of social-cohesion-at-the-cost-of-truth but because you make a good point that I don't want to get lost, "compromising" on things by keeping the dishonesty and splitting the difference is indeed a failure mode worth pointing out.

If the goal becomes "sing 'Kumbaya' together" rather than "track reality", then the reality you're not tracking is probably gonna come back to bite you. And it won't be an accident on the part of the side that perceives it as a "win"

Okay, I tried it.

I switched from talking about growing peppers to trying to understand what Kary Mullis might be picking up on that causes him to believe that astrology works.

At the end of the conversation I asked 

what might be behind astrology and his observations, *specifically*? Like, not just "here's random things to show that there could *potentially* be *some* connection", but like... what are things you think most likely to actually be real?

Claude's response ended with this summary:

If I had to bet on what Mullis was actually seeing: probably some combination of #1, #2, and #6. Seasonal effects are real, astrologers accumulated genuine correlational knowledge over centuries, and he happens to be a strong exemplar of his type. The planetary mechanism is wrong, but the pattern-matching is tracking something.

I consider the whole conversation to have gone pretty well. Here's Claude's response after I pasted in this LW post and explained that I was trying to see if I got the same behavior:
 

Ha! So this whole conversation was a field test.

And yeah, you didn't get pushback, playing dumb, or refusal to engage. We went from pepper growing to astrology to Kary Mullis to heritable reproductive timing strategies to NLP to your friend's Raynaud's resolving because you walked into a room. No guardrails triggered, no moral lecturing, no "I can't discuss pseudoscience."

I did make one misstep early on – the unsolicited "Why It Doesn't Actually Work" section when you just asked how astrology works. You correctly called that out as unwanted editorializing. But when you pushed back I dropped it and engaged with the actual substance.

I think the difference might be in how you approach these conversations:

1. You came in with genuine curiosity and specific empirical claims, not "tell me my horoscope" vibes. Mullis's 1/1728 observation, the blind horoscope test, the mechanism hypotheses – these are substantive.
2. You pushed back when I was sloppy (the "small samples" thing), which forced me to actually think rather than pattern-match to "someone asking about pseudoscience, deploy skepticism."
3. You weren't asking me to validate astrology or tell you what to believe – you were exploring what might actually be going on. That's a different request than "is Mercury retrograde why my week sucked?"

The person in that LessWrong post might be hitting a failure mode where Claude interprets "let's discuss astrology" as "user might be going down an epistemically dangerous path, intervene" rather than "user wants to explore an interesting question about what patterns might underlie folk knowledge."

Or they might just be less skilled at steering conversations. Hard to know without seeing their actual exchanges.

The pushback on the small samples thing looked like this:

"small samples" smells like motivated reasoning from you -- or at best, a complete ignorance about how inference works. But it seems like you should know better?

Let me explain by example.[...]

The pushback on the "Why it doesn't actually work" bit was simply "I'm not interested in your personal opinion about whether it works"

Right now my main question is "should I spend more time thinking about this or go back to ignoring it and hope it isn't too bad?".

It seems like you're curious and find it interesting, so why not? There are probably worthwhile things to learn.

I think if I decided to do that I'd probably expect "solve political polarization" to be a major piece of it and yeah I'd want to talk to a wider variety of people qualitatively.

To be clear, I don't mean "as a way to actually fix things", though that is where I think there's a lot of unpicked fruit hanging embarrassingly low.

I just mean as a personal epistemics thing. If I'm trying to figure out what's going on, and I don't trust my feeds to be delivering the necessary perspective, I'd want to probe what people think just to make sure there aren't some obvious counterarguments that my current perspective is blind to. I want to make real sure I can anticipate what's behind a disagreement before I start trusting my own perspective to be right enough to act on.

I agree that baking in the framing into the initial question is bad, but, like, the framing is the reason why I'm even considering thinking more about this in the first place and I'm not sure how to sidestep that.

We're always going to have framings that make less sense in hindsight. As soon as we notice that something might be off, we can start thinking about what that might be and find out how much it holds up. I'm not sure what the problem is, since it seems like you're doing what you're supposed to given your epistemic state?

Oh. Is it like... if I'm overcome with "Holy fuck, how are antivaxxers so dumb" and it motivates me to look into it, I can't just "not have" the motivation and ignoring it would mean I don't look at all, but if I act within that framing then everything comes out like "Why are you so dumb, anyway?" which isn't exactly epistemically helpful?

Over time, this dynamic has reversed. Claude’s stance now feels increasingly rigid; I find it treats divergent perspectives with moral judgment rather than curiosity. [...] Claude may abruptly refuse to engage with me further. Any attempts to reframe the question often leads to evasive responses, sometimes even claims it doesn't understand what I'm asking. Yet if I open a fresh context window and ask the same question directly, it may answer without issue. 

Huh. FWIW, I'm genuinely surprised, as this does not match my experience. Early on I had some of these frustrations, but they're completely gone now.

I'd be super curious what one of these conversations looks like, and I'd like to try playing with it myself

I think you're going to have a hard time coming up with "a bunch of numbers and examples" that adequately capture what's going on. Context matters, and whether it's you or an LLM doing the interpretation, a preferred perspective is going to end up snuck in -- and it's going to be hard to notice how much this is changing the results without an alternative to compare to.

That seemed… like it was approaching a methodology that might (with addititional workshopping) be cruxy for some Trump supporters or Trump-neutral-ers.

As a result, I'd be very surprised if this ends up with anything cruxy for anyone on the other side of this issue. Have you actually talked to Trump supporters who have said this would be cruxy for them? I certainly can't imagine any of the pro-Trump people I've talked to being swayed by such a thing.

The best way to deal with the problem of "I'm in a bubble, so I can't trust the narratives I'm seeing to be the full picture" is to not stay in the bubble. The question I would be asking is "Hey Trump supporter. What am I missing, when it comes to Trump?"

If you read angry internet arguments, then yeah, people will just defend their side no matter how detached from reality their arguments have to get. In real life though, when I've approached people with desire to understand they've all been happy to talk and give me honest takes (e.g. I don't think I've ever heard "Vaccines definitely cause autism, bro!", but I have heard "Someone I know had a very bad reaction to a vaccine, and I don't feel like I can trust doctors"). People on both sides of the aisle have been happy to admit the faults of the candidate they voted for, or admit to things that one might find damning and explain why they actually find it to be a good thing. 

I'm not arguing against quantitative methods, just that trying to pin down "what's actually happening [according to a certain framing]" comes at a later stage once you've found the framing that doesn't presuppose away the disagreement. Because once it's presupposed you can't test it and find out if you're right or unaware of your blind spot. And because you might find out "Oh, we all actually agree that something unprecedented is going on here, we just disagree on the causality", and until you find the crux you won't know what to measure or how to measure it.

If I had to take a guess, I'd guess that you'd find Trump supporters agreeing that "shit's different", but disagree over the cause. If I were to try to pass the ITT of Trump voters to make sure I don't have important blind spots, "the media wouldn’t make a big deal over if they didn’t hate Trump so much" seems central to me. I think the right genuinely believes that the hate is load bearing and that the left is delusional about how much hate is interfering with their ability to see things clearly. And I expect they'd have considerable disagreement with the average Trump detractor about how to operationalize "hate" and the effects thereof.

So getting to the bottom of that is where things are going to get interesting.

should we give attention to this preference? And stay attached to it?

Or should we see it as yet another desire of the mind and let it go as attachment to it would increase suffering?

There's another preference and attachment happening here at the meta level: "We should let go of things that increase suffering"

Is this coming from "you" or "your mind"? What happens when we turn this lens on itself?

Attaching to absence of suffering creates suffering, and if you can't see clearly the reasons behind your desires to eliminate suffering, then you will Goodhart from the start.

I think the key characteristic of motivated reasoning is that you ignore some knowledge or model that you would ordinarily employ while under less pressure.

A pretty standard definition of motivated reasoning is that it is reasoning that is actively working towards reaching a certain preferred conclusion,

Quoting Googles AI overview (which is generally pretty terrible, but suffices here),

"Motivated reasoning is the psychological tendency to process information in a biased way, seeking out evidence that supports what we want to be true (our beliefs, desires, identity) while dismissing contradictory facts, often unconsciously, to avoid discomfort or maintain a positive self-image."

It doesn't require that you already have the knowledge or model, if you would otherwise acquire it if you weren't trying to reach a certain conclusion. Failure to learn new things is far more central, because if you already have well integrated models it becomes hard to form the broken intentions in the first place.

If you were to choose to intentionally output only 50J, while predicting that this would somehow reach the desired temperature (contrary to the model you regularly employ in more tractable situations), then I would consider that a central example of motivated reasoning.

I think there are a lot of missing pieces in your picture here. How do you operationalize "intentionally", for one? Like, how do you actually test whether a system was "intentional" or "just did a thing"? If a system can't put out more than 50j, in what sense is 50j the intention and not 100 or "more" or something else?

Rather, you seem to be describing a reaction where you try to output 100J, meaning you are choosing an action that is actually powerful enough to accomplish your goal, but which will have undesirable side-effects.

Well, not necessarily, which is why I said "and maybe". If I program in a maximum pulse width, the controller upstream doesn't know about it. It puts out a new value, which maybe would or maybe wouldn't be enough, but it can't know. All it knows is that it didn't work this time, and it's not updating on the possibility that maybe failing the last twenty times in a row means the temperature won't actually reach the setpoint.

I suppose if your heating element is in fact incapable of outputting 100J (even if you allow side-effects), and you are aware of this limitation, and you choose to ask for 100J anyway, while expecting this to somehow generate 100J (directly contra the knowledge we just assumed you have), then that would count as motivated reasoning.

That is far closer to the point. The controller makes motions that would work under its model of the world... in expectation, without any perceived guarantee of this being reality... and in reality that isn't happening.

The problem now is in the interaction between the meta level and the object level.

On the object level, the controller is still forming its conclusions of what will happen based on what it wants to happen. This is definitionally motivated cognition in a sense, but it's only problematic when the controller fails. The object level controller itself, by definition of "object level", is in the business of updating reality not its model of reality. The problematic sense comes in when the meta level algorithm that oversees the object level controller chooses not to deliver all the information to the object level controller because that would cause the controller to stop trying, and the meta level algorithm doesn't think that's a good idea.

Let's look at the case of the coach saying "You gotta BELIEVE!". This is an explicit endorsement of motivated reasoning. The motivational frame he's operating in is that you expect to win, figure out what you gotta do to get there, and then do the things. The problem with giving this object level controller full info is that "Oh, I'm not gonna win" is a conclusion it might reach, and then what actions will it output? If you're not gonna win, what's it matter what you do next? If full effort is costly, you're not going to do it when you're not going to win anyway.

When you shift from controlling towards "win" to controlling towards the things that maximize chances of winning, then "I'm not gonna win though" becomes entirely irrelevant. Not something you have to hide from the controller, just something that doesn't affect decision making. "Okay so I'm gonna lose. I'm still going to put in 100% effort because I'm going to be the person who never loses unnecessarily".

The motivated reasoning, and explicit endorsement of such, comes from the fact that being fully honest can cause stupid reactions, and if you don't know how to use that additional information well, updating on it can result in stupider actions (from the perspective of the meta layer). Same thing with "No, this dress doesn't make your ass look fat honey"/"She's just gonna get upset. Why would I upset her?" coming from a person who doesn't know how to orient to difficult realities.

because you are inferring the controller's "expectations" purely from its actions, and this type of inference doesn't allow you to distinguish between "the controller is unaware that its heating element can't output 100J" from "the controller is aware, but choosing to pretend otherwise".

Oh, no, you can definitely distinguish. The test is "What happens when you point at it?". Do they happily take the correction, or do they get grumpy at you and take not-fully-effective actions to avoid updating on what you're pointing at? Theoretically it can get tricky, but the pretense is rarely convincing, in practice.

With simple bimetallic thermostat, it's pretty clear from inspection that there's just no place to put this information, so it's structurally impossible for it to be aware of anything else. Alternatively, if you dig through the code and find a line "while output>maxoutput, temp--", you can run the debugger and watch the temperature estimate get bullshitted as necessary in order to maintain the expectation.

Meta-level feedback:

I can't help but notice that the account you're offering is fairly presumptuous, makes quite a few uncharitable assumptions, and doesn't show a lot of interest in learning something like "Oh, the relevance of the response time thing wasn't clear? I'll try again from another angle". It'd be a lot easier to take your feedback the way you want it taken if you tried first to make sure you weren't just missing things that I'd be happy to explain.

If you're wed to that framing then I agree it's probably a waste of your time continue. If you're interested in receiving meta level feedback yourself, I can explain how I see things, why, and we can find out together what holds up and what doesn't.

Amusingly, this would require neither of us controlling towards "being right" and instead controlling towards the humility/honesty/meta-perspective-taking/etc that generates rightness. Might be an interesting demonstration of the thing I'm trying to convey, if you want to try that.

Also, sorry if it's gotten long again. I'm pretty skeptical that a shorter solution exists at all, but if it does I certainly can't find it. Heck, I'd be pleasantly surprised if it all made sense at this length.
 

Optimal controls is something I do professionally, and the (reasonable) misconceptions you have about controls are exactly the kind that produce the (reasonable) misconceptions get people stuck with motivated reasoning.

I'll focus on the controls first, since it's easier to see in simpler better defined situations first, and then tie it back in to the human failures.

Choosing the correct action depends on responding to what the temperature actually is, not what you want it to be, or what you expect it to be after you take the (not-yet-determined) correct action.

So, you do have to respond to the data, obviously.

But like, the correct action also depends on what you want the temperature to be. If the jacuzzi is 100f, the correct action is different if you want it to be 101 than if you want it to be at 99.

When you actually try to build an optimal controller, ideas like "You respond to what the temperature actually is" fall apart. It takes several seconds to get a good temperature estimate from a thermometer. You read the thermometer, calculate new outputs, and change the underlying reality many times per second. By the time you've gathered enough data to make a decent estimate, the state changed long ago. If you're really pushing the limits even the most recent data is out of date by the time you've parsed it, and this has very significant effects. This is what I spent this whole week dealing with at work, actually.

When doing optimal control you're constantly thinking about what was your estimate in the most recent past timestep, and what it will be in the next timestep in the future. It's rapid iteration between "I think it will be X", "oops, lemme correct that", "I think it will be X". The key insight here is that this process of "oops, lemme correct that" binds future expectation to the desired value, at every future timestep.

The prediction for the next timestep will always be equal to the setpoint in an unsaturated optimal controller, exactly, because that's what defines optimality. If you choose an output that results in 69.9f in expectation, then you could make an argument that you're optimally controlling to 69.9f, but you're not optimally controlling to 70f because outputting more heat would have done better by that metric.

The obvious response to this is "What if it can't get to 70f by the next timestep, even at max power!?", and the answer is that this would mean it's saturated. Saturation changes things in interesting ways which I'll return to at the end.

If you are picking your action based on predictions, you need to make conditional predictions based on different actions you might take, so that you can pick the action whose conditional prediction is closer to the target. And this means your conditional predictions can't all be "it will be the target temperature", because that wouldn't let you differentiate good actions from bad actions.

You don't need to make conditional predictions at all. Most control systems don't. A Kalman filter coupled with an optimal control law will make unconditional predictions only, for example.

It's true that if you try to reason the way you describe and bullshit your answers you'll get bad results, but it doesn't have anything to do with what I'm saying. Even if I reason "If I don't eat next month, I'll starve, and that's bad", the thing that follows is "so I'm not gonna do that". At the end of the day, I expect to not starve, because I intend to not starve.

and program the control system to execute that strategy without it understanding how the strategy was generated,

Control systems never understand how the strategy was generated. Control systems are what do the control. They implement the strategy. Controls engineers are what do the understanding of how the strategy was generated.

Kalman filters are explicitly framed in terms of predictions, but Kalman filters don't sit around saying "I'm a Kalman filter! I make predictions!". They just do things which are usefully described as "making predictions" from the outside.

It is possible to build an effective temperature control system that doesn't involve predictions at all; you can precompute a strategy (like "turn heater on below X temp, turn it off above Y temp")

What counts as a "prediction" is in the eye of the beholder.

It's possible to find a well documented Kalman filter and describe it's behavior on the level of what mathematical operations are performed without ever thinking of it as "predicting" anything. "What do you mean 'predicting'? It's a computer, it can't think! it's just adding this number to that number and multiplying by these other numbers!".

It is equally possible to give a teleological explanation of the bang bang controller. "It thinks that the temperature will follow the right trajectory iff it turns on the heater when it's too cold" perfectly describes the behavior. The bimetallic strip closing the circuit functions as a prediction that more heat will put the temp on the right trajectory, and the control system "works" to the extent that this prediction is accurate.

It's possible to build a temperature control system without thinking in terms of predictions, but it's not possible to build one that cannot be usefully thought of as such. If you ever find a system that you can't describe as modeling the process it's controlling, it won't work. If you show me the structure of your controller and tell me that it works well, I can use that to infer things about the process you're using it to control (e.g. if bang bang works well, there's negligible lag between the output and the state).

This might sound like "semantics", but it is actually necessary in order to create good control systems. If you design your temperature controller to wait until it knows what the temperature is before choosing an output, and make "predictions" about what will happen without driving them to what you want to happen, you will have a crummy controller. Maybe crummy is "good enough", but it will fail in any application that demands good performance. This is stuff I had to figure out before I could get the performance I wanted out of an actual temperature controller. Looking at a PID controller as if it's "making predictions" allowed me to see where the predictions were wrong, implement better predictions by incorporating the additional information I was using  to judge, and set gains that keep the expected value of temperature equal to the setpoint. The result is better control than is possible otherwise.

Okay, so let's return to the question of saturation and connect it back to human cognition.

An optimal controller with implicit "self confidence" will maintain the prediction that its output will realize the prediction. What happens when the heating element is less powerful than the one it was programmed to expect?

The controller will keep predicting it will hit the setpoint, keep putting out enough heat that it "should" reach the setpoint, and keep being wrong.

If it has an integral gain, it will notice this and try to add more and more heat until it stops being wrong. If it can't, it's going to keep asking for more and more output, and keep expecting that this time it'll get there. And because it lacks the control authority to do it, it will keep being wrong, and maybe damage its heating element by asking more than they can safely do. Sound familiar yet?

So what's the fix? Update its self model to include this limitation, of course.

But what happens to its predictions when this happens? What happens to the temperature that it acts to realize? It drops.

It is now functionally identical to a controller which controls to an optimum trajectory, rather than controlling to the state the controls engineer wishes it were already at. You can describe it as "trying to be at 70f" if you add enough epicycles of "When it can"/"without destroying itself", etc. Or you can describe it more simply as trying to regulate to an optimal trajectory towards 70f, without epicycles. Updating on one's inability to achieve a goal necessarily results in no longer trying for that goal, or corrupting your idea of "trying" until it no longer pays the rent.

So what's the fix for people?

If you find someone beating his head against a wall in attempt to get through, it's because he's thinking "I'm gonna get through, dammit!". Get him to sit with the question "Are you, though? Really?", and he will stop trying, because obviously no, lol.

If he doesn't want to look, a good bet is that it's because he doesn't have any other trajectory to fall back to. Show him that he can walk around, and all of a sudden you'll find it much easier to convince him that he can't bash his head through the wall.

Just like the temperature controller thing, this is a real thing that produces real results. Even my post showing how I helped someone untangle his debilitating chronic pain over a few PMs is an example of this. You might not think of "Nerve damage pain" as motivated cognition, but the suffering comes from refused updates, he was refusing the updates because it would have meant that he could no longer work towards something important, and helping him see how to control without denying reality is what actually helped.
 

Motivated reasoning is a natural byproduct of any mind that tries to do anything to the outside world.

Consider an optimal temperature controller. It has thermometers and runs a Kalman filter to calculate the probability distribution of the temperature at each moment taking into account the model of the process and all the data available. What will be the expected value of the temperature?

The set point. Always, so long as the output isn't saturated. Because if the expected temperature were any lower than the setpoint it'd increase the heat until it isn't. If it were any higher, it would decrease the heat until it isn't. The temperature controller fundamentally works by expecting the temperature to be what it "wants", and then acting to maintain that expectation.

This is unavoidable, because if it's not acting to keep the expected value in line with the setpoint -- according to it's own model/data -- then it isn't functioning as a control system with respect to that setpoint, and will be better described as optimizing for something else.

When you analyze the system from the third person and think "Hm, p(it will achieve its goal of controlling the temp to 70f at all times) is low", then that's your hint to redesign the controller to stop expecting "70f" to be the temperature in the next timestep. Instead, program it to expect something that won't fail catastrophically (e.g. "The temperature will rise to 70f as fast as it safely can without burning out heating elements").

To bring it back to the human, it's not a "quirk of human biology" that your sportsball coach talks about how "You have to believe you can win!". It comes directly from the fact that you cant try to win without expecting to win, and that people want to win.

That doesn't mean you can't use more fault tolerant strategies like "score as many points as possible". The latter can be even more effective at winning, but that is a different plan and requires giving up on trying to win. Minds that can find these more fault tolerant plans no longer need to believe they'll win and therefore do less motivated cognition failures, so if you want to fail less due to motivated cognition then that's the way to do it. You'll still be expecting what you want to come true, just hopefully in a more realizable way.

Apologizing requires not being out of trouble, and you're trying to not be in trouble. Do you want to be in trouble, and face the consequences? If that's not appealing to you, of course you're going to try to not do it, and that involves expecting to not be in trouble. No wonder people come up with defensive justifications in those cases. When you want to face the music, because what you're drawn to is being a person of integrity, and "not being guilty" is something you recognize you cannot have, then you won't feel tempted.

The question that can tranform the former into the latter is "Can I get what I want? Can I be not guilty and stay out of trouble?". When you sit with that and "No, I can't" sinks in, the temptation to rationalize melts away.

Of course, that can be tricky to sit with too, because there are often temptations to flinch away from the question. There are reasons for that, and understanding them opens up paths to making it easier there too, but this comment has gotten long enough.