Related to: Tsuyoku Naritai! (I Want To Become Stronger), Test Your Rationality, 3 Levels of Rationality Verification.
Robin and Eliezer ask about the ways to test rationality skills, for each of the many important purposes such testing might have. Depending on what's possible, you may want to test yourself to learn how well you are doing at your studies, at least to some extent check the sanity of the teaching that you follow, estimate the effectiveness of specific techniques, or even force a rationality test on a person whose position depends on the outcome.
Verification procedures have various weaknesses, making them admissible for one purpose and not for another. But however rigorous the verification methods are, one must first find the specific properties to test for. These properties or skills may come naturally with the art, or they may be cultivated specifically for the testing, in which case they need to be good signals, hard to demonstrate without also becoming more rational.
So, my question is this - what have you become reliably stronger at, after you walked the path of an aspiring rationalist for considerable time? Maybe you have noticeably improved at something, or maybe you haven't learned a certain skill yet, but you are reasonably sure that because of your study of rationality you'll be able to do that considerably better than other people.
This is a significantly different question from the ones Eliezer and Robin ask. Some of the skills you obtained may be virtually unverifiable, some of them may be easy to fake, some of them may be easy to learn without becoming sufficiently rational, and some of them may be standard in other disciplines. But I think it's useful to step back, and write a list of skills before selecting ones more suitable for the testing.
I've learned to recognize confused questions, and avoid wasting time answering them on their own terms when the proper response is to simply dissolve them. Considering this motivated me to drop a second major in philosophy which I was originally motivated to pursue because I enjoyed messing around with those confused questions, this has saved me quite a bit of time and effort.
I've greatly refined my ability to update on evidence and actually change my mind. Some of these beliefs have little payoff in terms of behavior, but occasionally it allows me to correct for fairly harmful mistakes.
Like Anna Salamon, my social skills have benefited.
I trust my explicit reasoning. I am able to agree or disagree with arguments separate from their conclusions. When I follow an argument, I can pinpoint if and where it contains elements I disagree with. I do not reach the end of arguments without spotting flaws in them and find myself disagreeing with the conclusions for reasons I cannot articulate. I am also prepared to spot and call out flaws in arguments whose conclusions I agree with, rather than treating them as soldiers on my side.
Although I consider myself to still be a novice in this ability, I have at least begun to be able to analyze the true motivations behind my actions, and what I want to want, and make efforts to shape my behavior with that knowledge.
I have to confess that I've learned very little of my core skills in rationality by reading and participating in Less Wrong though.