Nornagest comments on SotW: Be Specific - Less Wrong

37 Post author: Eliezer_Yudkowsky 03 April 2012 06:11AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (306)

You are viewing a single comment's thread. Show more comments above.

Comment author: Nornagest 04 April 2012 02:55:55AM 0 points [-]

A much better way would be to think of specific examples (of times when they were on time or late for an appointment, for instance).

This risks running into salience issues. Events which lead to failure or loss of status tend to be a lot more salient than those which lead to success, and while that by itself could still give you a reasonable if upsetting metric, I don't see any obvious way to control for differences in how this tendency manifests, or for differences in salience between domains (which could paradoxically lead to overweighting failures in a domain the testee considers personally important). Differences in general recall ability would also weight the results, but you could probably control for that with some extra effort.

Comment author: John_Maxwell_IV 04 April 2012 04:16:22AM 0 points [-]

You could probably fix these problems by specifically asking for recent examples, in the last few weeks. Also, note I might've picked an unusually upsetting personality question. If upsetting participants is a significant issue, choosing questions and personality metrics carefully could ameliorate things.

Also note that accurate personality assessment is not the primary goal here. In theory, it could be a good exercise even if it doesn't provide accurate personality assessment.

In fact, you could even discard scientific pretense altogether, and just have participants complete one of those "Which Harry Potter character are you?" type quizzes, but thinking of specific examples.