So when Holden Karnofsky says something like "rationality is a strong (though not perfect) predictor of success", maybe he is claiming that mental habits that make you better at figuring out what's true are actually quite useful in practice. (Or maybe by "rationality" he meains "instrumental rationality", in which case his statement would be true by definition.)

Of course, instrumental rationality is not a perfect predictor of success either. There are always stochastic factors with the potential to lead to bad outcomes. How strong a predictor it is depends on the size of such factors.